diff --git a/docs/jupyternotebook.md b/docs/jupyternotebook.md
new file mode 100644
index 00000000..18812af4
--- /dev/null
+++ b/docs/jupyternotebook.md
@@ -0,0 +1,178 @@
+This Jupyter Notebook set-up is intended for use with a linux system.
+
+***
+## Installing a Virtual Machine
+ For this example, VMWare will be used
+ Follow the instructions [here.](https://www.vmware.com/uk/products/workstation-player/workstation-player-evaluation.html)
+
+## Downloading Ubuntu
+ Download Ubuntu [here.](https://ubuntu.com/download/desktop)
+
+## Installing Ubuntu on Virtual Machine
+ Follow the installation instructions for Ubuntu [here.](https://theholmesoffice.com/installing-ubuntu-in-vmware-player-on-windows/)
+
+## Setting up Jupyter Notebooks with Python3 on Ubuntu
+
+ * Install Python3
+ For versions of Ubuntu 17.10 and above, Python3.6 is installed by default.
+ Check Python3 is installed with
+ `$ python3 --version`
+ To install Python3 for other operating systems, follow installation steps [here.](https://docs.python-guide.org/starting/installation/)
+
+ * Install appropriate Python package installer
+ Pip3 is recommended for Python package installation, it should already be installed if Python 3 >= 3.4 is installed.
+ To upgrade pip for Linux
+ `pip3 install -U pip`
+ `sudo apt install python3-pip`
+
+
+ * Install Jupyter
+ The following is one of the many ways to install jupyter, so feel free to install it your own way.
+
+ `$ pip3 install jupyter`
+ Add path to the `~/.bashrc` file to be able to run jupyter notebook from anywhere in terminal
+
+ `export PATH=$PATH:~/.local/bin`
+ Check jupyter notebook works
+
+ `$jupyter notebook`
+ This should open a web browser showing the Notebook Dashboard.
+ Close the page and proceed with the set-up.
+***
+## Installing TorQ and TorQ finance Started Pack (FSP) on Linux terminal
+* Create new directory and go into the new directory
+
+ `$ mkdir new_directory`
+ `$ cd new_directory`
+
+* Git clone links for TorQ and TorQ FSP from github
+ This step requires `git` to be installed.
+
+ `$ git clone https://github.com/AquaQAnalytics/TorQ.git`
+ `$ git clone https://github.com/AquaQAnalytics/TorQ-Finance-Starter-Pack.git`
+
+* Create new folder for your TorQ stack
+
+ `$ mkdir deploy`
+ `$ cd deploy`
+
+* Within the Deploy folder, copy TorQ into deploy and then copy everything not already in TorQ from FSP
+
+ `$ cp -r ../TorQ/* ./`
+ `$ cp -r ../TorQ-Finance-Starter-Pack/* ./`
+
+* Change port number in setenv.sh within the deploy folder
+
+ - Edit KDBBASEPORT number with appropriate text editor
+ - In the line `export KDBBASEPORT=6000`, change `6000` to a different number
+ - Save edited setenv.sh file
+
+* **Source the setenv file**
+
+ `$ source setenv.sh`
+ May show:Command 'curl' not found, follow the instructions to install curl
+***
+## Install q
+* Download and save appropriate q package from kx downloads from [here.](https://kx.com/connect-with-us/download/)
+ A pop up will appear for opening the appropriate zip file, open the file and extract it's contents into Downloads.
+* Copy q into home from downloads, if it's not already extracted there
+ `$cp - r q /home/$USER`
+* In home directory, get appropriate library for the download
+ - Check linux version (can do this anywhere within linux terminal)
+ `$uname -m` (this gives options depending on results)
+ I.e for 32-bit download, get the 32-bit library
+ `$ sudo apt-get install libc6-i386`
+ For more information on installing q check out [Installing kdb+.](https://code.kx.com/v2/learn/)
+* Check if q runs within the home directory (`$cd ~`)
+ `$ q/l32/q`
+ - Test that q is working
+ `q)til 6`
+ `0 1 2 3 4 5`
+ - exit q session
+ `q)\\`
+
+* Set alias for q (rlwrap must be installed for this step)
+ - edit the .bashrc file in your home directory
+ - Add following line to bottom of file:
+ `alias q='QHOME=~/q rlwrap -r ~/q/l32/q'`
+ - Save file and close text editor
+* Source .bashrc
+ `$ source .bashrc`
+ May show ```bash:need:not found / bash:alias:to:not found/bash:alias:enable:not found```, but you should be able to type q anywhere in the linux terminal and a q session will open up.
+ May also show `Command 'rlwrap' not found`, follow the instructions to install rlwrap
+***
+## Install modules for Jupyter Notebook
+ As there are a number of ways for installing python modules, there is a `jnbrequirements.txt` file provided which details the required modules for this notebook to run.
+ It is recommended to run the following
+ `pip3 install -U pip`
+ `sudo python3 -m pip install --force-reinstall pip==9.0.1`
+ `pip3 install -r jnbrequirements.txt --user`
+ As Python modules are frequently modified, there is no guarantee that the module versions used at the time jnbrequirements.txt was written will work at a later time.
+***
+## User Credentials
+ Open the `credentials.csv` and add in your host, username and password
+ Note your host, username and password must be **comma separated**
+i.e host,username,password
+ localhost,admin,admin
+***
+## Setting up routine Jupyter Notebook e-mails
+
+ To set up the conversion of Jupyter Notebooks to HTML and send them to a specified e-mail(s)
+ - Edit the `JUPYTEREMAIL` and `JUPYTERLOC` path variables in the `jupyterrun.sh` script
+ - To find where the jupyter-nbconvert command is run from, run `$ which jupyter-nbconvert` from the command line, copy the output and set is as the `JUPYTERLOC` variable
+ The path to `jupyter-nbconvert` needs to be manually added, as the location of the file may change based on what installer is used
+ Mailutils will also have to be installed, to do this run
+ `$ apt-get install mailutils`
+
+ - Bash the `jupyterrun.sh` script
+
+ If the user agrees to setting up a cronjob, an e-mail containing the HTML version of the notebook checks, will be sent everyday at 9am.
+ For more information on how to set up crontabs, check out: [This link.](https://crontab.guru/)
+ A `no crontab for test1` output may show, in this case type `$ crontab -e` and select an editor. The cronjob should appear when the editor opens. Save and exit the file.
+***
+## Running Jupyter Notebooks Manually
+To run Jupyter Notebooks manually use
+ `$ jupyter-notebook`
+
+ The notebook will automatically choose a free port number or you can run
+ `$ jupyter-notebook --port xxxx`
+ Replace xxxx with a port number of your choice
+***
+## Adding more Jupyter Notebook checks for your processes
+ To add additional checks in the notebook for processes other than tickerplant1, rdb1 and hdb1, copy the appropriate code and change the names of the processes
+ For example: Adding an additional rdb check
+ - Reveal the hidden code with the button at the top of the notebook
+ - Under `RDB Results` heading, copy the code in the cell and paste to a new cell
+
+ - Change `rdb1` in the yellow boxes to your process name and run the cell (SHIFT+ENTER)
+
+## Troubleshooting
+- When installing Ubuntu in VM, there may be an error with the Intel VT/AMD V virtualisation in BIOS
+ Follow instructions [here](https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/sect-Virtualization-Troubleshooting-Enabling_Intel_VT_and_AMD_V_virtualization_hardware_extensions_in_BIOS.html
+) to enable this virtualisation
+
+- When setting an alias for q rlwrap must be downloaded
+ Check if rlwrap is downloaded with:
+ `$rlwrap -v`
+ If rlwrap is installed, rlwrap followed by a version number should be returned
+ If rlwrap is not installed run:
+ `$ sudo apt install rlwrap`
+
+- When running `. torq.sh summary`, a nohup Exit 127 error may show.
+ In the home directory (cd ~), edit the `./.profile` file and add the following to the end of the file and save
+ `export PATH=$PATH:"~/q/l32/q"`
+ source the .profile file
+ `$ source .profile`
+
+ Edit the `/etc/environment file`, by adding the appropriate path to the q/l32/q file and save
+ `:/path/to/q/l32/"`
+ Source the /etc/environment file
+ `$ source /etc/environment`
+
+- If port numbers aren't showing when running . torq.sh summary
+ Install netstat with:
+ `$apt install net-tools`
diff --git a/jupyternb/credentials.csv b/jupyternb/credentials.csv
new file mode 100644
index 00000000..ff31614b
--- /dev/null
+++ b/jupyternb/credentials.csv
@@ -0,0 +1,2 @@
+host,username,password
+localhost,admin,admin
diff --git a/jupyternb/jnbchecks.ipynb b/jupyternb/jnbchecks.ipynb
new file mode 100644
index 00000000..fc17458d
--- /dev/null
+++ b/jupyternb/jnbchecks.ipynb
@@ -0,0 +1,489 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Process Checks \n",
+ " \n",
+ " This notebook will run default process checks on Tickerplant, RDB and HDB. \n",
+ " Processes are queried via qconnection with the username and password supplied in the credentials.csv file. \n",
+ " Port numbers are supplied from `. torq.sh summary` table. \n",
+ "- - - -"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from IPython.display import HTML\n",
+ "HTML('''\n",
+ "The raw code for this IPython notebook is by default hidden for easier reading.\n",
+ "To toggle on/off the raw code, click here.''')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Tickerplant Results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#importing required modules\n",
+ "from qpython.qconnection import QConnection as qcon\n",
+ "from qpython.qcollection import QDictionary as qdict\n",
+ "from contextlib import redirect_stdout as rd_so\n",
+ "from datetime import datetime\n",
+ "from datetime import date\n",
+ "import time\n",
+ "import psutil\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "import matplotlib.pyplot as plt\n",
+ "%matplotlib inline\n",
+ "\n",
+ "#set qcon variables\n",
+ "import csv\n",
+ "with open('credentials.csv') as csv_file:\n",
+ " csv_reader = csv.reader(csv_file, delimiter=',')\n",
+ " line_count = 0\n",
+ " for row in csv_reader:\n",
+ " if line_count == 0:\n",
+ " print(f'Credentials required are {\", \".join(row)}')\n",
+ " line_count += 1\n",
+ " else:\n",
+ " host=row[0]\n",
+ " un=row[1]\n",
+ " pswd=row[2]\n",
+ " print(f'\\t host:{row[0]} username: {row[1]} password: {row[2]}.')\n",
+ " line_count += 1\n",
+ " print(f'Processed {line_count} lines.') "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Process Summary\n",
+ "\n",
+ "Table showing process name, status, PID, Port numbers, CPU and Memory usage.\n",
+ "* Process status indicated by colours green (up) and red (down).\n",
+ "* Killtick, tpreplay1 and compression1 should usually have a down status indicated."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": false
+ },
+ "outputs": [],
+ "source": [
+ "#run torq summary\n",
+ "torq_sum= !\"${TORQHOME}\"/torq.sh summary\n",
+ "\n",
+ "#convert array to numpy array\n",
+ "summary=np.asarray(torq_sum)\n",
+ "\n",
+ "#split each element of array by | character\n",
+ "sum_list = [i.split('|') for i in summary]\n",
+ "df = pd.DataFrame(sum_list)\n",
+ "\n",
+ "# take first row and use as header for df\n",
+ "new_header = df.iloc[0]\n",
+ "df = df[1:]\n",
+ "df.columns = new_header\n",
+ "\n",
+ "#trim whitespace from headers\n",
+ "cols=[]\n",
+ "for i in df.columns:\n",
+ " cols.append(str.strip(i))\n",
+ "df.columns=cols\n",
+ "\n",
+ "#trim whitespace from all objects within dataframe\n",
+ "data = df.select_dtypes(['object'])\n",
+ "df[data.columns] = df.apply(lambda x: x.str.strip())\n",
+ "\n",
+ "#function to extract status of processes - takes a string argument\n",
+ "def stat_proc(process):\n",
+ " process_info = df.loc[df['PROCESS'] == process]\n",
+ " STAT = process_info['STATUS'].astype(str)\n",
+ " return STAT\n",
+ "\n",
+ "# function to extract the port number for each process - takes a string argument\n",
+ "def find_portno(process):\n",
+ " process_info = df.loc[df['PROCESS'] == process]\n",
+ " PORT = process_info['PORT'].astype(str).astype(int)\n",
+ " return PORT\n",
+ "\n",
+ "# function to extract the port number for each process - takes a string argument\n",
+ "def find_pid(process):\n",
+ " process_info = df.loc[df['PROCESS'] == process]\n",
+ " PID = process_info['PID'].astype(str).astype(int)\n",
+ " return PID\n",
+ "\n",
+ "#function to decode bytes to strings\n",
+ "def byte_decode(table,cols):\n",
+ " table[cols] = table[cols].applymap(lambda x: x.decode('utf-8'))\n",
+ "\n",
+ "#function to print mem and cpu stats\n",
+ "def print_mem_stats(pid):\n",
+ " with open(\"tmp.txt\",\"a\") as file:\n",
+ " with rd_so(file):\n",
+ " return (psutil.Process(pid)).memory_percent()\n",
+ "def print_cpu_stats(pid):\n",
+ " with open(\"tmp.txt\",\"a\") as file:\n",
+ " with rd_so(file):\n",
+ " return (psutil.Process(pid)).cpu_percent(interval =1.0)\n",
+ "\n",
+ "if stat_proc(\"tickerplant1\").tolist() == [\"down\"]:\n",
+ " print(\"Please check if the following processes are all up: Tickerplant1, rdb1, hdb1\")\n",
+ "else:\n",
+ " #return PIDs\n",
+ " df.replace('', np.nan, inplace=True)\n",
+ " pids = df[\"PID\"]\n",
+ " pids = pids.dropna()\n",
+ " pids = [int(i) for i in pids]\n",
+ " \n",
+ " #create array of mem and cpu stats \n",
+ " mems=[]\n",
+ " cpus=[]\n",
+ " \n",
+ " for pid in pids:\n",
+ " mems.append(print_mem_stats(pid));\n",
+ " cpus.append(print_cpu_stats(pid))\n",
+ " \n",
+ " #insert 0 into cpu/mem for processes that are down \n",
+ " nopid = df[df['PID'].isnull()].index.tolist()\n",
+ " for i in range(len(nopid)):\n",
+ " mems.insert((nopid[i]-1),0)\n",
+ " cpus.insert((nopid[i]-1),0)\n",
+ " \n",
+ " #append mem and cpu onto summary table \n",
+ " df['%MEM']=mems\n",
+ " df['%CPU']=cpus\n",
+ " df=df.round({'%MEM':1})\n",
+ " \n",
+ " \n",
+ "# set colour on down processes to red and up processes to green\n",
+ "def colour_down_red(col):\n",
+ " color = 'red' if 'down' in col else 'green'\n",
+ " return 'color: %s' % color\n",
+ "df.style.applymap(colour_down_red, subset=['STATUS'])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Count of tables in Tickerplant \n",
+ "\n",
+ "Table to show the count in each table found in the Tickerplant \n",
+ "* The counts in each of these tables should be 0, as the Tickerplant should not be storing any data.\n",
+ "* If the counts in any of these tables is not 0, this could indicate a slow subscriber."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"tickerplant1\").tolist() == [\"up\"]:\n",
+ " with qcon(host, port=find_portno('tickerplant1'), username=un, password=pswd,timeout=3.0) as q:\n",
+ " #counts tickerplant tables\n",
+ " tablecounts = q(\"enlist tables[]!count each value each tables[]\", pandas=True)\n",
+ " display(tablecounts)\n",
+ "else:\n",
+ " print('The Tickerplant process is down. Unable to count tables in Tickerplant')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Tickerplant Log file size increasing\n",
+ "\n",
+ "Checks if log messages in the log file of the tickerplant is increasing.\n",
+ " * If log messages are increasing, the tickerplant is receiving data.\n",
+ " * If log messages are not increasing, the tickerplant may not be recieving data. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"tickerplant1\").tolist() == [\"up\"]:\n",
+ " with qcon(host, port=find_portno('tickerplant1'), username=un, password=pswd,timeout=3.0) as q:\n",
+ " #counts log file size is increasing \n",
+ " log1=(q(\"hcount .u.L\"))\n",
+ " time.sleep(2)\n",
+ " log2=(q(\"hcount .u.L\"))\n",
+ " print (\"Log file sizes are increasing: \", log1 0])\n",
+ "else:\n",
+ " print('The Tickerplant process is down. Unable to check process handles connected to Tickerplant.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# RDB Results"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"rdb1\").tolist() == [\"up\"]:\n",
+ " with qcon(host, port=find_portno('rdb1'), username=un, password=pswd,timeout=3.0) as q:\n",
+ " #Check tables in rdb are same as tables in tickerplant \n",
+ " tables = q('all 1_tables[] in ((exec w from .servers.SERVERS where proctype=`tickerplant)0)(\"tables[]\")')\n",
+ " #Check count of tables in rdb - data is being sent from the tickerplant \n",
+ " tptordb = q('enlist tables[]!count each value each tables[]', pandas=True)\n",
+ " time.sleep(3)\n",
+ " tptordb2 = q('enlist tables[]!count each value each tables[]', pandas=True)\n",
+ " #Check that data in table can be queried\n",
+ " rdbtquery=q('5#select from last tables[]', pandas=True)\n",
+ " byte_decode(rdbtquery,['sym'])\n",
+ " tabname=q('last tables[]')\n",
+ " #Check that only data from today is present in the rdb tables\n",
+ " onedatet = q('enlist tables[]!{last exec distinct time.date from x}each tables[]', pandas=True)\n",
+ "else:\n",
+ " print('The RDB process is down.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### RDB tables\n",
+ "\n",
+ "Checks if the tables in the rdb are the same as the tables in the Tickerplant."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"rdb1\").tolist() == [\"up\"]:\n",
+ " print (\"RDB tables are same as Tickerplant tables : \", tables)\n",
+ "else:\n",
+ " print('Unable to check if tables in RDB are the same as Tickerplant tables.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### RDB table counts\n",
+ "\n",
+ "Checks to see if data is being sent from the Tickerplant to the RDB\n",
+ "* Indicates whether counts in RDB tables are increasing over time (3 second period)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"rdb1\").tolist() == [\"up\"]:\n",
+ " tptordb.set_index(tptordb.columns.tolist())\n",
+ " tptordb2.set_index(tptordb2.columns.tolist())\n",
+ " rdb_comp = tptordb < tptordb2\n",
+ " # set colour; True = Green; False = Red\n",
+ " def rdbtable_colour(val):\n",
+ " color = 'IndianRed' if val ==False else 'DarkSeaGreen'\n",
+ " return 'background-color: %s' % color\n",
+ " display(rdb_comp.style.applymap(rdbtable_colour))\n",
+ "else:\n",
+ " print('Unable to count RDB tables.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### RDB Query\n",
+ "Checks to see if tables in the RDB can be queried "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"rdb1\").tolist() == [\"up\"]:\n",
+ " print ('Table to be queried:')\n",
+ " name = tabname.decode('UTF-8')\n",
+ " print (name)\n",
+ " display(rdbtquery)\n",
+ "else:\n",
+ " print('Unable to query RDB tables.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### RDB Date Checks\n",
+ "Check to see what what date(s) are in tables \n",
+ "* Green indicates that only today's date is in the RDB table\n",
+ "* Red indicates there may be more than one date or no date found in the table"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"rdb1\").tolist() == [\"up\"]:\n",
+ " onedatet_col=onedatet.fillna(\"No date\")\n",
+ " def rdbtable_colour(val):\n",
+ " color = 'IndianRed' if val != date.today() else 'DarkSeaGreen'\n",
+ " return 'background-color: %s' % color\n",
+ " display(onedatet_col.style.applymap(rdbtable_colour))\n",
+ "else:\n",
+ " print('Unable to check date in RDB tables.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# HDB Results\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"hdb1\").tolist() == [\"up\"]:\n",
+ " with qcon(host, port=find_portno('hdb1'), username=un, password=pswd,timeout=10.0) as q:\n",
+ " #Check hdb table counts excl. eod_summary and eod_summary_iex\n",
+ " hdbtablecount=q('raze{0!select table:x,cnt:count i by date from x where date >=.z.d-5}each tables[] except `heartbeat`logmsg', pandas=True)\n",
+ "else:\n",
+ " print('The HDB process is down.')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### HDB Table Counts\n",
+ "* Counts for heartbeat and logmsg should be 0\n",
+ "* If counts for other tables are 0, no data has been recieved for the day before"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "if stat_proc(\"hdb1\").tolist() == [\"up\"]:\n",
+ " if isinstance(hdbtablecount, pd.DataFrame):\n",
+ " byte_decode(hdbtablecount,['table'])\n",
+ " display(hdbtablecount.set_index(hdbtablecount.columns.tolist()))\n",
+ " else:\n",
+ " print('There is no data in the HDB')\n",
+ "else:\n",
+ " print('Unable to count HDB tables.')"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/jupyternb/jnbrequirements.txt b/jupyternb/jnbrequirements.txt
new file mode 100644
index 00000000..2e1fbbf8
--- /dev/null
+++ b/jupyternb/jnbrequirements.txt
@@ -0,0 +1,109 @@
+apturl==0.5.2
+asn1crypto==0.24.0
+attrs==19.3.0
+backcall==0.1.0
+bleach==3.1.0
+Brlapi==0.6.6
+certifi==2018.1.18
+chardet==3.0.4
+command-not-found==0.3
+cryptography==2.1.4
+cupshelpers==1.0
+cycler==0.10.0
+decorator==4.4.1
+defer==1.0.6
+defusedxml==0.6.0
+distro-info===0.18ubuntu0.18.04.1
+entrypoints==0.3
+html5lib==0.999999999
+httplib2==0.9.2
+idna==2.6
+importlib-metadata==1.3.0
+ipykernel==5.1.3
+ipython==7.10.2
+ipython-genutils==0.2.0
+ipywidgets==7.5.1
+jedi==0.15.1
+Jinja2==2.10.3
+jsonschema==3.2.0
+jupyter==1.0.0
+jupyter-client==5.3.4
+jupyter-console==6.0.0
+jupyter-core==4.6.1
+keyring==10.6.0
+keyrings.alt==3.0
+kiwisolver==1.1.0
+language-selector==0.1
+launchpadlib==1.10.6
+lazr.restfulclient==0.13.5
+lazr.uri==1.0.3
+louis==3.5.0
+macaroonbakery==1.1.3
+Mako==1.0.7
+MarkupSafe==1.0
+matplotlib==3.1.2
+mistune==0.8.4
+more-itertools==8.0.2
+nbconvert==5.6.1
+nbformat==4.4.0
+netifaces==0.10.4
+notebook==6.0.2
+numpy==1.17.4
+oauth==1.0.1
+olefile==0.45.1
+pandas==0.25.3
+pandocfilters==1.4.2
+parso==0.5.2
+pexpect==4.2.1
+pickleshare==0.7.5
+Pillow==5.1.0
+prometheus-client==0.7.1
+prompt-toolkit==2.0.10
+protobuf==3.0.0
+psutil==5.6.7
+ptyprocess==0.6.0
+pycairo==1.16.2
+pycrypto==2.6.1
+pycups==1.9.73
+Pygments==2.5.2
+pygobject==3.26.1
+pymacaroons==0.13.0
+PyNaCl==1.1.2
+pyparsing==2.4.5
+pyRFC3339==1.0
+pyrsistent==0.15.6
+python-apt==1.6.4
+python-dateutil==2.6.1
+python-debian==0.1.32
+pytz==2018.3
+pyxdg==0.25
+PyYAML==3.12
+pyzmq==18.1.1
+qPython==2.0.0
+qtconsole==4.6.0
+reportlab==3.4.0
+requests==2.18.4
+requests-unixsocket==0.1.5
+SecretStorage==2.3.1
+Send2Trash==1.5.0
+simplegeneric==0.8.1
+simplejson==3.13.2
+six==1.11.0
+system-service==0.3
+systemd-python==234
+terminado==0.8.3
+testpath==0.4.4
+tornado==6.0.3
+traitlets==4.3.3
+ubuntu-drivers-common==0.0.0
+ufw==0.36
+unattended-upgrades==0.1
+urllib3==1.22
+usb-creator==0.3.3
+wadllib==1.3.2
+wcwidth==0.1.7
+webencodings==0.5.1
+widgetsnbextension==3.5.1
+xkit==0.0.0
+zipp==0.6.0
+zope.interface==4.3.2
diff --git a/jupyternb/jupyterrun.sh b/jupyternb/jupyterrun.sh
new file mode 100644
index 00000000..08d82693
--- /dev/null
+++ b/jupyternb/jupyterrun.sh
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+#set user email variable
+JUPYTEREMAIL="putyouremailhere@example.com"
+
+#set variable for jupyter-nbconvert command path
+JUPYTERLOC=$HOME/jupyter/jupyter_torq/bin/jupyter-nbconvert
+
+#set TORQJUPYTER variable path
+TORQJUPYTER=${TORQHOME}/jupyternb
+
+#set JNBCRONSCRIPT variable
+JNBCRONSCRIPT=${TORQJUPYTER}/jnbcronjob.sh
+
+#if JNBCRONSCRIPT does not exist, create empty file
+if [[ ! -f ${JNBCRONSCRIPT} ]];then
+ touch ${JNBCRONSCRIPT}
+fi
+
+#set jupyter notebook variable
+JNOTEBOOK=${TORQJUPYTER}/jnbchecks.ipynb
+
+#set variable for HTML version of jupyter notebook
+JUPYTERHTML=${TORQJUPYTER}/jnbchecks.html
+
+#Create script to be run by cron job
+#this generates the commands to execute the notebook,
+#convert to html and mail to the required user email
+echo "#!/bin/bash
+
+#Convert jupyter notebook to HTML version
+${JUPYTERLOC} --execute --to html ${JNOTEBOOK}
+
+#Email HTML version of notebook to user email
+echo \"New Jupyter Notebook Generated\" | mail -A ${JUPYTERHTML} -s \"New Jupyter Notebook Generated\" ${JUPYTEREMAIL}
+
+#Remove most recent notebook, so previous notebooks are not sent
+rm ${JUPYTERHTML}" > ${JNBCRONSCRIPT}
+
+#User prompt to whether they wish to create a crontab or schedule it themselves
+read -p "Would you like to create a crontab job to e-mail the notebook.html everyday at 9 [y/n]?" answer
+
+if [[ "$answer" = "y" ]] ; then
+ #generate the line to be run in crontab itself
+ if crontab -l | grep -q 'jnbcron';then
+ echo "Crontab exists"
+ else
+ echo "Crontab does not exist. Creating crontab..."
+ (crontab -l; echo -e " BASH=/bin/bash\n TORQHOME=${TORQHOME}\n * 9 * * * cd ${TORQHOME}; bash ${JNBCRONSCRIPT}") | crontab -
+ fi
+else
+ echo "User has decided not to create a crontab"
+
+fi
+