PYCOF

1. AWS credentials setup

Some functions such as pycof.sql.remote_execute_sql(), pycof.data.f_read() or pycof.misc.write() may need to have access to AWS to intereact with different services. On a AWS instance (e.g. EC2, SageMaker), you may not need to got though these steps as the instance would directly consume the IAM role you assigned to it.

Then, on a local instance (which will rely on a IAM user), run:

aws configure

and enter your AWS access and private keys.

Note that this setup is required only if you use AWS services such as S3 or Redshift. These functions also work without credentials for local use.

2. PYCOF configuration setup

The functions pycof.sql.remote_execute_sql() and pycof.format.send_email() will, by default, look for the credentials located in /etc/.pycof/config.json. On Windows, save the config file as C:/Users/<username>/.pycof/config.json.

The file follows the below structure:

{
"DB_USER": "",
"DB_PASSWORD": "",
"DB_HOST": "",
"DB_PORT": "3306",
"DB_DATABASE": "",
"__COMMENT_1__": "Email specific, send_email",
"EMAIL_USER": "",
"EMAIL_SENDER": "",
"EMAIL_PASSWORD": "",
"EMAIL_SMTP": "smtp.gmail.com",
"EMAIL_PORT": "587",
"__COMMENT_2__": "IAM specific, if connection='SSH' in remote_execute_sql",
"CLUSTER_NAME": "",
"__COMMENT_3__": "SSH specific",
"SSH_USER": "",
"SSH_KEY": "",
"SSH_PASSWORD": ""
}

On Unix based system, run:

sudo nano /etc/.pycof/config.json

and paste the above json after filling the empty strings (pre-filled values are standard default values).

Reminder: To save the file, with nano press CTRL + O, confirm with ENTER then CTRL + X to exit.

SQL

Data

Formatting