Fai Ho Fu's Blog: April 2019

Monday, 29 April 2019

AWS Secrets Manager python script

"Introduction to AWS Secrets Manager"
https://www.aws.training/learningobject/video?id=19441
The link above is to a 15 minute introductory video that covers the "AWS Secrets Manager" service.
The summary from this training and certification page is:
"In this course, we offer a brief overview of AWS Secrets Manager, highlighting the integration with Amazon Relational Database Service. The video discusses how this service can help manage secrets like OAuth tokens and application API keys, and concludes with a demonstration of how AWS Secrets Manager can generate and automatically rotate your secrets and passwords."
A similar python script that Bridgid used in her training and certification video is:

print "secrets manager test"
#!/usr/bin/python
import MySQLdb
import boto3
import json
#trying to connect to secrets manager
secretmanager = boto3.client('secretsmanager', endpoint_url='https://secretmanager.us-west-2.amazonaws.com')/
secret_id = 'db/test/DevelopmentDB' #this is the secret name within aws-SecretManager
db_user = json.loads(secretmanager.get_secret_value(SecretId=secret_id)['SecretString'])['username']
db_password = json.loads(secretmanager.get_secret_value(SecretId=secret_id)['SecretString'])['password']
db = MySQLdb.connect(host='secrets-manager--demo.abc123.us-east-1.rds.amazonaws.com', #db endpoint
            user=db_user, #db username
            passwd=db_password, #db username password
            db="secretManager") #db name
# you must create a cursor object, it will let you execute all the queries needed
cur = db.cursor()
cur.execute("select * from employee")
# print all the first cell of all the rows
print "Rows from the test database"
for row in cur.fetchall():
    print row[0] # index 0 equates to the first field in the record
db.close()

There’s many youtube video’s covering AWS Secrets Manager:
https://www.youtube.com/user/AmazonWebServices/search?query=secrets+manager

Thursday, 4 April 2019

Talend Admin Center TAC login

Below is the location of the configuration.properties file.
Viewing this file is very handy when locked out of the TAC gui.
The default login for TAC (after a fresh install) is
username: security@company.com
password: admin
To confirm what the default login is, it’s best to refer to the configuration.properties file on the TAC server, as detailed below.
On a local vbox installation, the Talend installation was in the local home directory.
Via a terminal session, the commands to navigate to this directory, and view the file is:
(your installation may be in a different directory, but the path should be similar.
Talend may be a different value dependant on your installation
tac may be a different value dependant on your installation)

cd ~/Talend/tac/apache-tomcat/webapps/tac/WEB-INF/classes
vi configuration.properties

Wednesday, 3 April 2019

REST API–A to Z

This article contains a summary of notes related to what REST API’s are.

(Care of https://www.smashingmagazine.com/2018/01/understanding-using-rest-api/)

REST API : REpresentational State Transfer Application Programming Interface.

A request to a URL which is the REST API endpoint results in a response containing data which is called the resource.

Anatomy

The endpoint (or route)

this is the url, which is made up of a root-endpoint, and path and finally the query
for example “http://faihofu.blogspot.com/search/label/aws”
”www.faihofu.blogspot.com” is the root-endpoint
”/search/label/aws” is the path
another example “https://www.google.co.uk/search?q=faihofu”
”www.google.co.uk” is the root-endpoint
”/search” is the path
”q=faihofu” is the query

The method
the headers
The data (or body or message)

1 Endpoints

When referring to API documentation, the path me be stated as
“/search/label/:labelvalue”
Any colons within the path denote a variable, which need to be replaced when hitting the endpoint.

2 Method

These are the types of API requests:

This is the default method to request / read from the endpoint / server

Post

This creates a new resource / data on the endpoint server

This updates the resource / data on the endpoint server

Patch

see PUT

Delete

This deletes resource / data on the endpoint server

These methods are used to perform one of the actions (CRUD):

Create
Read
Update
Delete

3 Headers

“Headers are used to provide information to both the client and server.
You can find a list of valid headers on MDN’s HTTP Headers Reference.”

HTTP Headers are property-value pairs that are separated by a colon.
The example below shows a header that tells the server to expect JSON content.
You can send HTTP headers with curl through the -H or --header option.
To view headers you’ve sent, you can use the -v or --verbose option as you send the request, like this:

curl -H "Content-Type: application/json" https://api.github.com -v

4 Data, Body, Message

The data (sometimes called “body” or “message”) contains information you want to be sent to the server.
This option is only used with POST, PUT, PATCH or DELETE requests.
To send data through cURL, you can use the -d or --data option:

curl -X POST <URL> -d property1=value1 -d property2=value2

Authentication

To perform a basic authentication with cURL, you can use the -u option, followed by your username and password, like this:

curl -x POST -u "username:password" https://api.github.com/user/repos

HTTP Status codes and error messages

When a CURL command returns an error (use --verbose to view full details), the error code can be translated using the HTTP status reference:
MDN’s HTTP Status Reference.

CURL (cURL) https://curl.haxx.se/

CURL is a command line tool and library used to transfer data, and in this blog-context, can be used to access / hit API endpoints.

To test if curl is installed, run the following command:

curl –version

Using query parameters

When accessing an endpoint using query parameters, the “?” and “=” need to be escaped in the url.

For example

curl https://api.something.com/somepath\?query\=queryvalue

JSON

API responses are often returned in JSON format.

JSON = JavaScript Object Notation

JSON documents can contain non-structured data, in the form of key-value pairs.

{
  "property1": "value1",
  "property2": "value2",
  "property3": "value3"
}

Example curl commands

curl -u "username:password" https://api.github.com/user/repos -d "{\"name\":\"testrepo12345\"}"

curl -u "username:password" https://api.github.com/repos/{username}}/testrepo12345 DELETE 

curl -u "username:password" https://api.github.com/user/repos

Search