diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
index 69f26d3..4a5bdfb 100644
--- a/.github/workflows/lint.yml
+++ b/.github/workflows/lint.yml
@@ -11,6 +11,7 @@ permissions:
jobs:
lint:
+ name: Lint Python Code
runs-on: ubuntu-latest
steps:
@@ -21,6 +22,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.14'
+ cache: 'pip'
- name: Install dependencies
run: pip install -r helpers/requirements.txt
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
new file mode 100644
index 0000000..609f658
--- /dev/null
+++ b/.github/workflows/test.yml
@@ -0,0 +1,28 @@
+name: Run Python Unit Tests
+
+on:
+ push:
+ branches: [ main, dev ]
+ pull_request:
+ branches: [ main ]
+
+jobs:
+ test:
+ name: Run Unit Tests
+ runs-on: ubuntu-latest
+
+ steps:
+ - name: Checkout Code
+ uses: actions/checkout@v6
+
+ - name: Set up Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: '3.14'
+ cache: 'pip'
+
+ - name: Install dependencies
+ run: pip install -r helpers/requirements.txt
+
+ - name: Run Unit Tests
+ run: python runtest.py
diff --git a/.gitignore b/.gitignore
index b9e3cbd..8906a3d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
__pycache__
-*.xlsx
\ No newline at end of file
+*.xlsx
+!test_output.xlsx
\ No newline at end of file
diff --git a/documentation.md b/documentation.md
index 5da6412..874bf8b 100644
--- a/documentation.md
+++ b/documentation.md
@@ -2,32 +2,204 @@
The goal of this file is to provide a higher level overview of how this script works to help identify issues, improvements, as well as future maintainability.
## At a Glance
There are two primary methods that this script runs through: The CLI version and the GUI version. The CLI version is more or less function complete in how I intend for it to function and operate, though it still has some rough edges, especially in a user interaction factor. The GUI version is what will continue to receive updates and more improvements over time. These two versions can be toggled between in the `config.ini` file.
+---
## main.py
This file serves as the starting point for the whole script. If the GUI mode is activated, it will start the GUI from `gui.py`, otherwise, it will take an input from the user and pass it over to `sheetmaker.py` if you're making a new sheet or to `modes.py` if you're using the other features.
## analytics.py
Anything related to handling data from Alma gets handled through this script. Between setting up the browser, parsing the HTML, and handling inputs for things such as the SQL.
+### get_columns
+This function is where all the information related to generating SQL templates is stored. I found it much easier to track things in lists of dictionaries, with each dictionary containing a "Key" value for the section of the database it is searching, as well as "Cols" or columns which stores a list of the values that are listed under the associated "Key" value. Giving this function different values while calling returns different templates for different purposes. Due to how some listings do not appear while some columns are present, some of these are broken up into really small pieces.
+### get_col_len
+Returns the total number of columns within the SQL template. This helps with tracking how many columns need to be searched across while reading the tables within Alma analytics.
+### get_table
+Handles reading and parsing out the table information from the Alma screen.
+### pull_one_search
+Takes a list of MMS IDs and returns a list of the same size of `True` or `False`. False indicates that NO OneSearch listings appeared under the MMS ID search, while True indicates that something did return. This aids in finding out which MMS IDs and listings are still in circulation.
+### pull_data
+Given some data, this function automates creating a SQL statement and pulling data from the database.
+### input_sql
+Automates opening the SQL input window in Alma and inputting the given text.
+### check_element
+Aids in automating checking whether a given HTML element is present on the page, to know when it is okay to move to the next step.
+### click_element
+A template for telling the web driver what to click given some details about the specific HTML element.
+### process_new_isbn
+UNFINISHED and UNUSED; This function would ideally be used to find what ISBNs we do own for particular books from the bookstore, where a different ISBN is provided.
+### process_analytics
+Function that interacts with the other scripts to aid in pulling the analytics data given the web driver. Contains all the logic for how to process each set of information.
+### setup_analytics
+Creates and sets up the web browser to be prepared for automation tasks.
+### setup_sql
+Given a set of keys and columns to read, creates a complete SQL statement to use.
+### export_analytics
+Exports the data to a given csv file path.
+### import_analytics
+Imports the data from the given csv file path.
+
## bookstore.py
Most of this code is pulled from the original bookstore data puller that was made to automate getting bookstore data. It will have the user pass a CAPTCHA and then have them select data to pull from to get the specific term of textbook data they are interested in.
+### str_clean
+Removes certain special characters and whitespace from strings, aids in standardizing book titles.
+### get_page_soup
+Uses the BS4 library to grab a given URL, take the HTML contents and turn it into a workable form of data.
+### get_link
+Helper function to reduce number of replace statements later, simply helps create a link to use for price comparisons.
+### get_prices
+UNFINISHED and UNUSED; If there were to be a cost analysis function, this is what it would be, but the volume of books makes it a little too difficult to query this webserver as much as we would like to so it remains commented out.
+### pull_textbook_data
+The master information pulling function for the bookstore data.
+### pull_info
+Imports data from the bookstore csv file, mainly used by other parts of the program.
+
## classes.py
Stores the frameworks and methods for interacting with storing and processing book related data.
+### Book (Class)
+- Contstructor
+Creating an object requires a dictionary of a couple different values: Related course, section, instructor name and email, enrollment information, the ISBN, book title, author, edition, publisher, requirements state, requisition date, the bookstore comment, and the Alma Analytics data for the given book. The constructor will parse out the information to make it more easily accessible for processing later.
+- add_course
+When another entry for a book is found that already exists as an object, the new course information gets added via this function.
+- add_section
+Similar to the course entry, but instead adding a section to an existing course within an existing book.
+- add_isbn
+Adding ISBN values for different variants of the same book.
+- add_enroll
+Putting in additional enrollment information into the book to track all the campuses and possible enrollment values.
+- add_required
+If at any point a bookstore listing has the book as "Required", it sets the book to that status using this function.
+### get_max_index
+Finds the index of the course with the most sections within it in order to aid in reducing column count, as well as ensuring the largest courses are always first (leftmost).
+### get_max_courses
+Gets the total number of course sections to have (the number of courses the book with the most courses has).
+### get_max_sections_list
+Finds the number of sections needed for each course section. As an example, if we have two books, one book has 3 course with 10, 5, and 3 sections respectively ([10, 5, 3]), the other has 2 courses, one with 7 sections, the other with 6 sections ([7, 6]), then this function finds we need Course 1 to have 10 section slots, Course 2 to have 6, and Course 3 to have 3.
+### process_book
+Takes the preset information about a book and checks against all books so far to ensure that it does not already exist, otherwise it adds it to the master list.
+### process_courses
+Uses the prior get_max_courses function in order to create the headers and add them to the dataframe.
+### process_sections
+Uses the prior get_max_sections_list function in order to create the individual section headers and adds them to the dataframe.
+### process_isbns
+This function both finds the max number of ISBN columns required and puts the necessary columns into the dataframe.
+### import_data
+Imports all the data from the master book object list and imports it into the dataframe once it is ready to go.
+
## emails.py
Handles compiling and creating Excel sheets for PowerAutomate emails.
+### Book (Class)
+This class is a very tiny version of the other class that just takes in some basic information for the purposes of emails.
+- Constructor
+Takes the title, author, edition, year published, and access information.
+### Instructor (Class)
+- Constructor
+Takes in the instructor name, email, as well as course, section, and book information per section.
+- add_book
+Adds books to a given course section for each professor.
+### update_excel
+Updates the main sheet with marking off what emails have been successfully created.
+### create_email_excel
+Processes all the Instructor data to create the table to be used to write it into an Excel sheet.
+### write_to_excel
+Writes the email data to an Excel sheet to be used with PowerAutomate.
+
## enrollment.py
Handles pulling the relevant data from the enrollment csv file in order to get the maximum enrollment values.
+### get_enrollment_data
+Processes the format of the given CORE report enrollment file, returning campus and enrollment information.
+
## grabber.py
Deals with setting up and handling pulling data from the Outlook browser to get emails for individual professors.
+### process_name
+### process_suggestion
+### get_email
+### setup_grabber
+### grabber_gui
+- set_email_store
+- run_process_suggestion
+- run_check_ui
+- run_check_web
+- run_suggestion_ui
+- run_get_email
+### email_importer
+### email_exporter
+
## gui.py
This hosts all of the primary interactive GUI for the user.
+### GUI (Class)
+- Constructor
+- reset_main
+- print_main
+- build_main
+- build_emails
+- build_headers
+- build_advanced
+- build_sheet_outlook
+- build_sheet_alma
+- build_sheet_final
+- build_import_csv
+- start_analytics_csv
+- start_bookstore_csv
+- start_grabber_csv
+- start_mode
+- write_cfg
+- write_headers
+- write_emails
+### start_app
+
## helpergui.py
This is a much slimmer and simpler version of the main GUI class, to be something much more modular and additive.
+### AddedGUI (Class)
+- Constructor
+- reset
+- add_label
+- add_button
+### make_window
+
## modes.py
Aids in handling the various functions of the script, modularizing individual pieces into useful functions.
+### start_mode
+### csv_mode
+### email_mode
+### update_mode
+### emails_csv
+### analytics_csv
+### enrollment_update
+### analytics_update
+### emails_update
+### get_import
+
## output.py
-This script has a single function: outputting an Excel sheet!
+This script has a single function: outputting an Excel sheet! This is where formatting and such gets handled (i.e. color, column sizing, etc.)
+### write_to_sheet
+
## sheetmaker.py
Performs all of the functions related to creating new sheets from scratch.
+### make_excel_sheet
+
## utilties.py
This script hosts helpful functions that might be purposeful in multiple places around the various helper and main scripts. On top of this, it also helps to host hard coded data that is not necessary to keep in a configuration file (such as header values!). The organization is to help cut down on lines of code in other places, as well as keep information consistency so updating one variable does update it in all relevant places when needed.
+### get_int
+Takes a value, gets an integer out of it if it can, otherwise returns None.
+### get_clean
+Removes preset phrases and terms from the names of books and authors to reduce the number of duplicate listings.
+### get_state
+Takes strings of "True" or "False" and converts them to bool values.
+### get_directory
+### get_filepath
+### get_letter
+### get_edition_string
+### get_format_headers
+### get_replace_header
+### get_split_course
+### get_input
+### get_enabled
+### get_sheet_headers
+### get_config_headers
+### get_string_cleaners
+### get_row_info
+### get_campus
+### set_col_format
+A
+---
## CSV Storage
In order to store all the data in a way that is accessible, quick, and aids in subsequent run times, all pulled data is compiled into `.csv` files, each with their own format. This makes it so we don't have to re-run the bookstore scraper, email grabber, or analytics scraper again every single time we wish to do something.
### analytics.csv
@@ -47,6 +219,7 @@ Data is formatted as following:
Instructor Name, Email
### enrollment.csv
This is just a CORE report exported as a csv for all courses in the desired term. This must be done outside of the script itself.
+---
## Config Files
### config.ini
Primary settings for the script, though some of these are redundant / only used by one half of the script.
diff --git a/helpers/analytics.py b/helpers/analytics.py
index b68303c..8b3ba3b 100644
--- a/helpers/analytics.py
+++ b/helpers/analytics.py
@@ -12,82 +12,6 @@
import ast
import threading
-"""
-SELECT "Bibliographic Details"."Author" saw_0,
-"Bibliographic Details"."Earliest Possible Publication Year" saw_1,
-"Bibliographic Details"."Title" saw_2,
-"Bibliographic Details"."Publisher" saw_3,
-"Bibliographic Details"."MMS Id" saw_4,
-"Bibliographic Details"."ISBN" saw_5,
-"Bibliographic Details"."Edition" saw_6,
-"Bibliographic Details"."Material Type" saw_7,
-"Bibliographic Details"."Resource Type" saw_8,
-"Edition Simplified"."Edition Simplified (Num)" saw_9,
-FROM "Digital Inventory"
-WHERE
-"Bibliographic Details"."ISBN" LIKE '%9781478651123%'
-"""
-
-"""
-SELECT "Bibliographic Details"."Author" saw_0,
-"Bibliographic Details"."Earliest Possible Publication Year" saw_1,
-"Bibliographic Details"."Title" saw_2,
-"Bibliographic Details"."Publisher" saw_3,
-"Bibliographic Details"."MMS Id" saw_4,
-"Bibliographic Details"."ISBN" saw_5,
-"Bibliographic Details"."Edition" saw_6,
-"Bibliographic Details"."Material Type" saw_7,
-"Bibliographic Details"."Resource Type" saw_8,
-"Representation Access Rights"."Access Right Name" saw_9,
-"Representation Access Rights"."Access Right Desc" saw_10,
-"Edition Simplified"."Edition Simplified (Num)" saw_11,
-FROM "Digital Inventory" WHERE "Bibliographic Details"."ISBN" LIKE '%'
-"""
-
-"""
-SELECT "Bibliographic Details"."Author" saw_0,
-"Bibliographic Details"."Earliest Possible Publication Year" saw_1,
-"Bibliographic Details"."Title" saw_2,
-"Bibliographic Details"."Publisher" saw_3,
-"Bibliographic Details"."MMS Id" saw_4,
-"Bibliographic Details"."ISBN" saw_5,
-"Bibliographic Details"."Edition" saw_6,
-"Bibliographic Details"."Material Type" saw_7,
-"Bibliographic Details"."Resource Type" saw_8,
-"Representation Access Rights"."Access Right Name" saw_9,
-"Representation Access Rights"."Access Right Desc" saw_10,
-"Edition Simplified"."Edition Simplified (Num)" saw_11,
-FROM "Digital Inventory" WHERE UPPER("Bibliographic Details"."TITLE") LIKE UPPER('%')
-"""
-
-"""
-SELECT
- "Bibliographic Details"."Title" saw_0,
- "Vendor Interface"."Vendor Name" saw_1,
- "Vendor Interface"."Interface Name" saw_2,
- "-- Bibliographic Details"."MMS Id" saw_3,
- "Vendor Interface"."Available" saw_4
- FROM "E-Inventory"
- WHERE
-UPPER("Bibliographic Details"."TITLE") LIKE UPPER('%CLIMATE CASINO%')
-"""
-
-# TODO
-# fixing missing isbn or mismatched values rom the bookstore
-# if nothing returns, looking up by the book title instead, comparing
-# author, publisher and edition number
-# use new isbn and information to backpush updating the book information
-
-# can search by isbn but needs to be without restrictive filters
-# then needs to add on the license filters, log where necessary
-# if nothing is found at ISBN, then needs to pivot to searching for book
-# still store data at old ISBN when exporting to CSV, but inlude alternative
-# ISBN information..? and then reinclude additional ISBN if not in original sheet?
-
-# new solution : E-Inventory ?
-# this should correct any problems and now it should be feasible to take out the representation
-# access right sql queries, which will make it easier to access the correct information
-
def get_columns(key="ebook"):
"""Gets the columns being used to construct the SQL query."""
@@ -113,7 +37,7 @@ def get_columns(key="ebook"):
]
overall_section = "Digital Inventory"
- if key == "access":
+ elif key == "access":
sql_columns = [
{
"Key": "Bibliographic Details",
@@ -209,12 +133,7 @@ def get_table(driver):
# TODO
-# this needs SEVERE checking
-# first thought is the search link could possibly be incorrect?
-# maybe searching via MMS ID is improper?
-# patterns kristin told me:
-# NONE of the Ebooks were caught, ONLY CDLs, but not ALL CDLs
-# Print books were ONLY from our MAIN collection
+# may need further testing
def pull_one_search(driver, mms_list):
"""Opens a OneSearch tab to double check that the MMS ID exists within Primo.
Takes a list and returns a list, though only used with single IDs right now."""
@@ -255,7 +174,8 @@ def pull_one_search(driver, mms_list):
def pull_data(driver, bib_section, bib_value, sql_key):
- """"""
+ """Given a section, value, and specific SQL template, grabs the
+ relevant data that is desired."""
section, sql_cols = get_columns(key=f"{sql_key}")
sql = setup_sql(section, sql_cols, f"{bib_section}")
sql = sql.replace("%", f"%{bib_value}%")
@@ -298,60 +218,6 @@ def pull_data(driver, bib_section, bib_value, sql_key):
return return_list
-def pull_ebook_access(driver, mms_id):
- """"""
- section, sql_cols = get_columns(key="access")
- sql = setup_sql(section, sql_cols, "MMS Id")
- sql = sql.replace("%", f"%{mms_id}%")
-
-
-def pull_analytics(driver, isbn_list, state):
- """Pulls the analytics information for the given ISBN list from the table."""
- return_list = []
- sql_section, sql_columns = get_columns(state)
- sql_statement = setup_sql(sql_section, sql_columns)
-
- for isbn in isbn_list:
-
- sql_text = sql_statement.replace("%", f"%{isbn}%")
- input_sql(driver, sql_text)
- tr_list = get_table(driver)
-
- if tr_list == []:
- return return_list
-
- cutoff = 3 + get_col_len(sql_columns)
- tr_list = tr_list[cutoff:]
-
- # storing entries for empty portions
- store_dict = {}
- # storing order for indexing
- store_list = []
- for dict in sql_columns:
- for col in dict["Cols"]:
- store_dict[col] = ""
- store_list.append(col)
-
- for tr in tr_list:
- td_list = tr.find_all("td")
- for td in td_list:
- try:
- td_id = td["id"]
- td_text = td.get_text()
- id_list = td_id.split("_")
- cat_id = get_int(id_list[5])
- column = store_list[cat_id]
- store_dict[column] = td_text
- except Exception as err:
- print(err)
-
- return_list.append(store_dict.copy())
-
- # print(f"{isbn_list} --> {return_list}")
-
- return return_list
-
-
def input_sql(driver, text=""):
"""Helper function that inputs the given text into the SQL input box."""
click_element(driver, "td", "title", "Edit SQL, XML and other technical details")
@@ -401,20 +267,6 @@ def click_element(driver, tag, selector, detail):
print(err)
-def setup_sql(sql_section, sql_list, bib_section="ISBN"):
- """Helper function to turn the SQL categories and lists into a full single query."""
- sql = "SELECT"
- idx = 0
- for dict in sql_list:
- section = dict["Key"]
- for col in dict["Cols"]:
- sql += f' "{section}"."{col}" saw_{idx},'
- idx += 1
- sql += f' FROM "{sql_section}" WHERE "Bibliographic Details"."{bib_section}"'
- sql += """ LIKE '%' """
- return sql
-
-
def process_new_isbn(driver, title, state):
"""Helper function to find what ISBN we do own for a particular book."""
check_list = pull_one_search(driver, [title])
@@ -474,13 +326,6 @@ def process_new_isbn(driver, title, state):
return None
-# TODO
-# TEST WITH 'CLIMATE CASINO' AS THE TITLE
-# CANNOT DO DIGITAL INVENTORY WITH THE LICENSE TYPES DUE TO PHYSICAL
-# need to work around the digital license problem first !
-# rework this to reprocess the digitial poritions first
-# and then ensure it is checking for Book - Electronic first
-# or something similar
def process_analytics(analytics_driver, isbn):
"""Fully processes a given ISBN number and pulling all the data from Alma + processing it."""
year = None
@@ -508,8 +353,6 @@ def process_analytics(analytics_driver, isbn):
if mms_id.strip() == "":
continue
- # TODO
- # double check implementation of this one search checker
check_list = pull_one_search(analytics_driver, [mms_id])
if check_list[0] is False:
print(
@@ -578,8 +421,6 @@ def process_analytics(analytics_driver, isbn):
if mms_id.strip() == "":
continue
- # TODO
- # double check implementation of this one search checker
check_list = pull_one_search(analytics_driver, [mms_id])
if check_list[0] is False:
print(f"Missing from Primo (EBOOK); skipping. {mms_id}")
@@ -633,6 +474,11 @@ def process_analytics(analytics_driver, isbn):
for section in process_list:
if "user".upper() in section:
users = get_int(section.split(" ")[0])
+
+ # adding a failstate to ensure these values are marked as unknown
+ # and can be easily removed via find & replace
+ if users == -1:
+ users = "?"
if mms_id not in data:
data[mms_id] = {
@@ -732,6 +578,20 @@ def context_window(text_list):
return driver
+def setup_sql(sql_section, sql_list, bib_section="ISBN"):
+ """Helper function to turn the SQL categories and lists into a full single query."""
+ sql = "SELECT"
+ idx = 0
+ for dict in sql_list:
+ section = dict["Key"]
+ for col in dict["Cols"]:
+ sql += f' "{section}"."{col}" saw_{idx},'
+ idx += 1
+ sql += f' FROM "{sql_section}" WHERE "Bibliographic Details"."{bib_section}"'
+ sql += """ LIKE '%' """
+ return sql
+
+
def export_analytics(path, info):
"""Exports the analytics data into a CSV."""
with open(path, "w", newline="") as csvfile:
diff --git a/helpers/bookstore.py b/helpers/bookstore.py
index c1279b9..c0f849d 100644
--- a/helpers/bookstore.py
+++ b/helpers/bookstore.py
@@ -11,12 +11,21 @@
from helpers.helpergui import AddedGUI
-def str_clean(str):
+def str_clean(value):
"""Removes special characters and whitespace from a string."""
- new_str = str.replace(" ", "")
- new_str = new_str.replace(" ", "")
- new_str = new_str.replace("\t", "")
- new_str = new_str.replace("\n", "")
+ original = value
+ new_str = ""
+ first = True
+ while new_str is not original:
+ if first:
+ first = False
+ else:
+ original = new_str
+ new_str = original.strip()
+ new_str = new_str.replace(" ", "")
+ new_str = new_str.replace(" ", " ")
+ new_str = new_str.replace("\t", "")
+ new_str = new_str.replace("\n", "")
return new_str
@@ -116,6 +125,8 @@ def get_prices(driver, table):
main_div = material_dict[isbn]
main_div.click()
# TODO
+ # this has been partially implemented but it cannot handle the number of
+ # requests this script would like to make, check other TODO or documentation
# this needs to pull the used and new price points from the beaverstore page
# https://osubeaverstore.verbacompare.com/comparison?id=2026-Summer__AEC__411__400
html = main_div.get_attribute("innerHTML")
@@ -294,7 +305,7 @@ def set_term_val(value):
if row_skip == 0:
# if at some point it would be desirable to add in book pricing
# this function doesn't work as is, but i stopped progress on it
- # seeing has the compare site has rate limiting, which is fair
+ # seeing as the compare site has rate limiting, which is fair
# otherwise, it would be best to simply link to the compare site
# in the sheet if anything i think
# row_text.append(STORE LINK HERE)
diff --git a/helpers/classes.py b/helpers/classes.py
index 4376bc5..d046976 100644
--- a/helpers/classes.py
+++ b/helpers/classes.py
@@ -6,7 +6,7 @@
class Book:
"""Object that stores relevant book information to be used to format and
configure a dataframe. Takes in a dictionary to construct the base
- object. See main.py for the book_info dictionary structure."""
+ object. See sheetmaker.py for the book_info dictionary structure."""
def __init__(self, info):
# bookstore information
@@ -57,7 +57,10 @@ def add_course(self, course, section, instructor, email, enroll):
def add_section(self, course, section, instructor, email, enroll):
"""Adds section information to book object."""
- idx = self.courses.index(course)
+ try:
+ idx = self.courses.index(course)
+ except ValueError as err:
+ print(f"Course Index not found: {err}")
self.sections[idx].append([section, instructor, email, enroll[0]])
self.sec_size[idx] += 1
self.add_enroll(enroll[1], enroll[0])
@@ -87,11 +90,13 @@ def add_required(self):
def get_max_index(len_list):
- """Gets the index of the largest course based on number of sections."""
+ """Gets a list of indices of the largest courses in order based on number of sections."""
output = []
temp = len_list.copy()
while max(temp) > 0:
+ # append the index of the largest course
output.append(temp.index(max(temp)))
+ # set that course size to 0 before going to the next
temp[output[-1]] = 0
return output
@@ -105,7 +110,7 @@ def get_max_courses(book_list):
def get_max_sections_list(book_list, course_amt):
- """Gets the list of section counts by course for a book."""
+ """Gets the values for how many section columns to put per course numbering."""
sec_len = []
for idx in range(course_amt):
sec_len.append(0)
@@ -120,7 +125,7 @@ def get_max_sections_list(book_list, course_amt):
def process_book(book_list, book_dict):
- """Processed incoming book information and crates / adds book
+ """Processed incoming book information and creates / adds book
information to the main list of book objects."""
book = None
for search_book in book_list:
diff --git a/helpers/emails.py b/helpers/emails.py
index 50e310f..1a7fe17 100644
--- a/helpers/emails.py
+++ b/helpers/emails.py
@@ -30,6 +30,9 @@ def __init__(self, name, email, course, section, book):
self.data = {course: {section: [book]}}
def add_book(self, course, section, book):
+ # if we have the course & section, add book to it
+ # if we have course, but not section, make new section with book
+ # if neither, make new course and section dictionary
if course in self.data and section in self.data[course]:
self.data[course][section].append(book)
elif course in self.data:
@@ -58,27 +61,19 @@ def update_excel(directory, data, sheet_name):
cell.value = datetime.datetime.now()
break
workbook.save(directory)
+ workbook.close()
-def write_to_excel(directory, export_data, sheetname):
- """Exports given data to given directory with given sheetname."""
- dataframe = pd.DataFrame(data=export_data)
- writer = pd.ExcelWriter(directory, engine="xlsxwriter")
- dataframe.to_excel(
- writer, sheet_name=sheetname, startrow=1, header=False, index=False
- )
- worksheet = writer.sheets[sheetname]
- max_row, max_col = dataframe.shape
- column_settings = []
- for header in dataframe.columns:
- column_settings.append({"header": header})
- worksheet.add_table(0, 0, max_row, max_col - 1, {"columns": column_settings})
- worksheet.set_column(0, max_col - 1, 12)
- writer.close()
+# the goal here should be to load book data into a book object,
+# then for each instance of an instructor for that book,
+# assign the book reference to them with the course + section
+# if instructor already exists, just add the other information
+# otherwise, make new instructor
# TODO
# rewrite this into smaller functions to break it apart a little
+# could easily be much more functionized, making it easier to work with
def create_email_excel(input_sheet=None, file_name=""):
"""Creates an excel sheet to be formatted for usage with PowerAutomate function."""
full_config = configparser.ConfigParser()
@@ -276,7 +271,8 @@ def create_email_excel(input_sheet=None, file_name=""):
# course[1]}.
'
# TODO
- # combining class codes ?
+ # combining class codes or just restructuring this whole thing to make it
+ # cleaner and easier to work with in the future
# book_dict = {book_str1: [section1, section2],
# book_str2: [section1], etc.}
@@ -299,10 +295,10 @@ def create_email_excel(input_sheet=None, file_name=""):
book_str += ""
- print(book.access)
+ # print(book.access)
for access_data in book.access:
link = book.access[access_data]["Link"]
- print(book.access[access_data])
+ # print(book.access[access_data])
if access_data == "Ebook":
user_num = book.access[access_data]["Number"]
@@ -406,12 +402,21 @@ def create_email_excel(input_sheet=None, file_name=""):
write_to_excel(output_path, final_data, "Email List")
update_excel(input_path, book_idx, process_sheet)
- print("Email sheet exported. Main sheet updated.")
+ # print("Email sheet exported. Main sheet updated.")
-# the goal here should be to load book data into a book object,
-# then for each instance of an instructor for that book,
-# assign the book reference to them with the course + section
-
-# if instructor already exists, just add the other information
-# otherwise, make new instructor
+def write_to_excel(directory, export_data, sheetname):
+ """Exports given data to given directory with given sheetname."""
+ dataframe = pd.DataFrame(data=export_data)
+ writer = pd.ExcelWriter(directory, engine="xlsxwriter")
+ dataframe.to_excel(
+ writer, sheet_name=sheetname, startrow=1, header=False, index=False
+ )
+ worksheet = writer.sheets[sheetname]
+ max_row, max_col = dataframe.shape
+ column_settings = []
+ for header in dataframe.columns:
+ column_settings.append({"header": header})
+ worksheet.add_table(0, 0, max_row, max_col - 1, {"columns": column_settings})
+ worksheet.set_column(0, max_col - 1, 12)
+ writer.close()
diff --git a/helpers/enrollment.py b/helpers/enrollment.py
index f8a8a80..b3ead43 100644
--- a/helpers/enrollment.py
+++ b/helpers/enrollment.py
@@ -26,7 +26,7 @@ def get_enrollment_data(dir, instructor_dict={}):
elif section not in enrollment_dict[f"{course}"]:
enrollment_dict[f"{course}"][f"{section}"] = [max_enroll, campus]
else:
- print("I don't think these cases exist; will write the logic")
+ print("I don't think these cases exist (Multiple enrollments for course and section)")
# instructor things
instructor = df["PRIMARY_INSTRUCTOR"][idx]
diff --git a/helpers/grabber.py b/helpers/grabber.py
index 9a60006..df3895b 100644
--- a/helpers/grabber.py
+++ b/helpers/grabber.py
@@ -280,7 +280,11 @@ def run_get_email(name):
while True:
time.sleep(0.7)
try:
- # TODO trying to implement webdriverwait
+ # TODO
+ # implementing webdriverwait to speed up checking times
+ # and make this whole process a little faster and more precise
+ # otherwise, leaving this alone for now as it seems to cause more
+ # issues that it aids in fixing
# suggestion_box = WebDriverWait(driver, 3).until(
# EC.presence_of_element_located((By.CLASS_NAME, "ms-FloatingSuggestionsList-container"))
# )
diff --git a/helpers/test.py b/helpers/test.py
new file mode 100644
index 0000000..9e36e44
--- /dev/null
+++ b/helpers/test.py
@@ -0,0 +1,1002 @@
+import unittest
+import os
+import random
+import pandas as pd
+import openpyxl
+import datetime
+import configparser
+
+import helpers.analytics as alma
+import helpers.bookstore as book
+import helpers.classes as classes
+import helpers.emails as emails
+import helpers.enrollment as enroll
+import helpers.grabber as outlook
+import helpers.gui as gui
+import helpers.helpergui as helpgui
+import helpers.modes as modes
+import helpers.output as output
+import helpers.utilities as util
+
+from warnings import simplefilter
+
+# skipping selenium oriented functions as they require logging in via the interface
+# this could be done in headless mode w/ more looking over the HTML of the log in
+# pages, but focusing on the core logic functions for now
+
+class AnalyticsTest(unittest.TestCase):
+ """Testing the analytics.py file."""
+ def setUp(self):
+ # error message setup
+ self.null_msg = "get_columns: Null % SQL Section."
+ self.empty_msg = "get_columns: Empty % SQL Section."
+ self.missing_msg = "get_columns: Missing Key or Column values in % SQL."
+ self.sql_statements = ["SELECT", "FROM", "WHERE", "LIKE"]
+
+ # helper functions
+
+ # checks the SQL columns for proper dictionary setup
+ def check_columns(self, columns):
+ check = True
+ for dict in columns:
+ if not dict["Key"] or not dict["Cols"]:
+ check = False
+ break
+ return check
+
+ # functionizing the error messages
+ def get_sql_msg(self, value):
+ new_null = self.null_msg.replace("%", value)
+ new_empty = self.empty_msg.replace("%", value)
+ new_missing = self.missing_msg.replace("%", value)
+ result = {"Null": new_null, "Empty": new_empty, "Miss": new_missing}
+ return result
+
+ # testing get_columns
+ def test_ebook_sql(self):
+ section, columns = alma.get_columns(key="ebook")
+ messages = self.get_sql_msg("Ebook")
+
+ self.assertIsNotNone(section, messages["Null"])
+ self.assertTrue(section, messages["Empty"])
+
+ col_check = self.check_columns(columns)
+ self.assertTrue(col_check, messages["Miss"])
+
+ def test_access_sql(self):
+ section, columns = alma.get_columns(key="access")
+ messages = self.get_sql_msg("Access")
+
+ self.assertIsNotNone(section, messages["Null"])
+ self.assertTrue(section, messages["Empty"])
+
+ col_check = self.check_columns(columns)
+ self.assertTrue(col_check, messages["Miss"])
+
+ def test_physical_sql(self):
+ section, columns = alma.get_columns(key="physical")
+ messages = self.get_sql_msg("Physical")
+
+ self.assertIsNotNone(section, messages["Null"])
+ self.assertTrue(section, messages["Empty"])
+
+ col_check = self.check_columns(columns)
+ self.assertTrue(col_check, messages["Miss"])
+
+ def test_einventory_sql(self):
+ section, columns = alma.get_columns(key="e-inventory")
+ messages = self.get_sql_msg("E-Inventory")
+
+ self.assertIsNotNone(section, messages["Null"])
+ self.assertTrue(section, messages["Empty"])
+
+ col_check = self.check_columns(columns)
+ self.assertTrue(col_check, messages["Miss"])
+
+ def test_public_sql(self):
+ section, columns = alma.get_columns(key="public")
+ messages = self.get_sql_msg("Public")
+
+ self.assertIsNotNone(section, messages["Null"])
+ self.assertTrue(section, messages["Empty"])
+
+ col_check = self.check_columns(columns)
+ self.assertTrue(col_check, messages["Miss"])
+
+ # testing get_col_len
+ def test_col_len(self):
+ # example from real code
+ # expects 14 columns
+ columns = [
+ {
+ "Key": "Bibliographic Details",
+ "Cols": [
+ "Author",
+ "Title",
+ "ISBN",
+ "Resource Type",
+ "MMS Id",
+ "Earliest Possible Publication Year",
+ ],
+ },
+ {
+ "Key": "Physical Item Details",
+ "Cols": [
+ "Material Type",
+ "Num of Items (In Repository)",
+ "Temporary Physical Location in Use",
+ ],
+ },
+ {
+ "Key": "Holdings Details",
+ "Cols": ["Holdings Lifecycle", "Suppressed from Discovery"],
+ },
+ {"Key": "Edition Simplified", "Cols": ["Edition Simplified (Num)"]},
+ {"Key": "Location", "Cols": ["Location Name"]},
+ {"Key": "Temporary Location", "Cols": ["Temporary Location Name"]},
+ ]
+ num = alma.get_col_len(columns)
+ expected = 14
+ self.assertEqual(num, expected, f"get_col_len: Miscounting column count; expected {expected}, got {num}")
+
+ # selenium functions would go here
+
+ # testing setup_sql
+ def test_setup_sql(self):
+ col_keys = ["ebook", "access", "physical", "e-inventory", "public"]
+ for key_val in col_keys:
+ section, columns = alma.get_columns(key=key_val)
+ sql = alma.setup_sql(section, columns)
+ for command in self.sql_statements:
+ self.assertTrue(command in sql, f"setup_sql: {key_val} SQL statement missing {command}")
+
+ # export and import functions would go here
+
+
+class BookstoreTest(unittest.TestCase):
+ """Testing the bookstore.py file."""
+ def setUp(self):
+ self.clean_msg = "str_clean: Value mismatch from expected; % =/= ^"
+
+ def get_clean_msg(self, res, exp):
+ out_msg = self.clean_msg.replace("%", res)
+ out_msg = out_msg.replace("^", exp)
+ return out_msg
+
+ # testing str_clean
+ def test_clean_nl(self):
+ new_line = "\nsome text\n\n "
+ result = book.str_clean(new_line)
+ expected = "some text"
+ message = self.get_clean_msg(result, expected)
+ self.assertEqual(result, expected, message)
+
+ def test_clean_tab(self):
+ tab_char = "\t\thello \t world"
+ result = book.str_clean(tab_char)
+ expected = "hello world"
+ message = self.get_clean_msg(result, expected)
+ self.assertEqual(result, expected, message)
+
+ def test_clean_space(self):
+ space_text = " something a "
+ result = book.str_clean(space_text)
+ expected = "something a"
+ message = self.get_clean_msg(result, expected)
+ self.assertEqual(result, expected, message)
+
+ def test_clean(self):
+ text = " \t\n\n something \t t here \t "
+ result = book.str_clean(text)
+ expected = "something t here"
+ message = self.get_clean_msg(result, expected)
+ self.assertEqual(result, expected, message)
+
+ # get_page_soup selenium function
+
+ # testing get_link
+
+ # get_prices selenium function
+
+ # pull_textbook_data uses selenium
+
+ # pull_info import function
+ def test_pull_info(self):
+ directory = ["testing", "test_bookstore.csv"]
+ path = get_directory(directory)
+ table = book.pull_info(path)
+
+ term = "2022-Summer"
+ subject = "MTH : Mathematics"
+ code = "101"
+ section = "001"
+ instructor = "LastName, FirstName"
+ title = "Intro to Math Textbook"
+ edition = "11"
+ author = "Textbook Author"
+ isbn = "978-1-11-111"
+ publisher = "Textbook Publisher"
+ req = "Optional"
+ sku = "1"
+ comments = "Some Comment"
+ requisition = "2/2/2022 2:22:22 PM"
+
+ example_row = [term, subject, code, section, instructor, title, edition, author, isbn, publisher, req, sku, comments, requisition]
+ row = table[0]
+
+ for idx, cell in enumerate(row):
+ self.assertEqual(cell, example_row[idx], f"pull_info: {cell} doesn't equal {example_row[idx]}")
+
+
+class ClassesTest(unittest.TestCase):
+ """Testing the classes.py file."""
+ def setUp(self):
+ # individiual preset testing
+ self.analytics_data = {"Physical MMS Id": {
+ "Types": ["Physical"],
+ "Copies": [2],
+ "Users": [0],
+ "CDL": [False],
+ "Link": "https://www.example.com",
+ "Year": 2024,
+ "Location": "Valley Library BIB"
+ },
+ "Ebook MMS Id": {
+ "Types": ["Electronic"],
+ "Copies": [0],
+ "Users": [4],
+ "CDL": [True],
+ "Link": "https://www.example2.com",
+ "Platform": "",
+ "Year": 2020
+ }}
+ self.book_info = {
+ "Title": "Book Title",
+ "Author": "Book Author",
+ "Edition": "Book Edition",
+ "Instructor": "Example Name",
+ "Email": "Example Email",
+ "Course": "Example Course",
+ "Section": 1,
+ "Enroll": [1, "C"],
+ "ISBN": 1,
+ "Publisher": "Book Publisher",
+ "Req": "Optional",
+ "RequiDate": "Requisition Date",
+ "Comment": "Example Comment",
+ "Analytics": self.analytics_data,
+ }
+ self.book = classes.Book(self.book_info)
+
+ def generate_book_list(self):
+ # randomized testing
+ self.book_list = []
+ self.course_list = []
+ self.section_list = []
+
+ # course_templates = ["MTH", "BI", "WGSS", "CHEM", "GEO"]
+ section_templates = [1, 100, 200, 300, 400, 500]
+ campus_templates = ["C", "D", "N", "B", "Z", "L", "H", "PDX"]
+ platform_templates = ["OverDrive", "EBSCO", "Elsevier", "ProQuest", ""]
+ book_count = get_random(5, 100)
+ for num in range(1, book_count + 1):
+ base_course = f"Course 0"
+ base_section = section_templates[get_random(0, len(section_templates) - 1)]
+ base_enroll = [get_random(0, 150), campus_templates[get_random(0, len(campus_templates ) - 1)]]
+ edition = get_random(0, 20)
+ # sometimes the edition value is left empty
+ if edition == 0:
+ edition = None
+ title = f"Book Title {num}"
+ author = f"Book Author {num}"
+ instructor = f"Example Name {num}"
+ email = f"Example Email {num}"
+ publisher = f"Book Publisher {num}"
+ req_check = get_random(1, 2)
+ if req_check == 2:
+ requirements = "Required"
+ else:
+ requirements = "Optional"
+
+ base_analytics = {}
+ rand = get_random(1, 3)
+ if rand >= 2:
+ base_analytics[f"{num}Physical"] = {
+ "Types": ["Physical"],
+ "Copies": [get_random(1, 11)],
+ "Users": [0],
+ "CDL": [False],
+ "Link": "https://www.example3.com",
+ "Year": 2000 + get_random(0, 25),
+ "Location": "Valley Library BIB"
+ }
+
+ rand = get_random(1, 3)
+ if rand >= 2:
+ rand = get_random(1, 2)
+ if rand >= 2:
+ is_cdl = True
+ else:
+ is_cdl = False
+ base_platform = platform_templates[get_random(0, len(platform_templates) - 1)]
+ base_analytics[f"{num}Electronic"] = {
+ "Types": ["Electronic"],
+ "Copies": [0],
+ "Users": [get_random(1, 11)],
+ "CDL": [is_cdl],
+ "Link": "https://www.example4.com",
+ "Platform": base_platform,
+ "Year": 2000 + get_random(0, 25)
+ }
+
+ base_info = {
+ "Title": title,
+ "Author": author,
+ "Edition": edition,
+ "Instructor": instructor,
+ "Email": email,
+ "Course": base_course,
+ "Section": base_section,
+ "Enroll": base_enroll,
+ "ISBN": num,
+ "Publisher": publisher,
+ "Req": requirements,
+ "RequiDate": "2/2/2022 2:22:22 PM",
+ "Comment": "Example Comment",
+ "Analytics": base_analytics,
+ }
+
+ new_book = classes.Book(base_info)
+
+ book_sections = []
+
+ # generating sections for the already existing base section
+ section_count = get_random(1, 30)
+ for sec in range(1, section_count + 1):
+ course_section = section_templates[get_random(0, len(section_templates) - 1)]
+ sec_num = course_section + sec
+ sec_enroll = [get_random(0, 150), campus_templates[get_random(0, len(campus_templates) - 1)]]
+ new_book.add_section(base_course, sec_num, instructor, email, sec_enroll)
+
+ book_sections.append(section_count + 1)
+
+ # generate remaining courses + sections
+ course_count = get_random(1, 6)
+ for course in range(1, course_count + 1):
+ course_name = f"Course {course}"
+ course_section = section_templates[get_random(0, len(section_templates) - 1)]
+ course_enroll = [get_random(0, 150), campus_templates[get_random(0, len(campus_templates) - 1)]]
+ new_book.add_course(course_name, course_section, instructor, email, course_enroll)
+
+ section_count = get_random(1, 30)
+ for sec in range(1, section_count + 1):
+ sec_num = course_section + sec
+ sec_enroll = [get_random(0, 150), campus_templates[get_random(0, len(campus_templates ) - 1)]]
+ new_book.add_section(course_name, sec_num, instructor, email, sec_enroll)
+
+ book_sections.append(section_count + 1)
+
+ book_sections = sorted(book_sections, reverse=True)
+ self.section_list.append(book_sections)
+
+ # need to add one for the initial set
+ self.course_list.append(course_count + 1)
+ self.book_list.append(new_book)
+
+ # Book Class testing
+
+ # add_course method
+ def test_add_course(self):
+ course_name = "Test Course"
+ section_num = 999
+ instructor = "Test Name"
+ email = "Test Email"
+ enroll_data = [99, "D"]
+
+ self.book.add_course(course_name, section_num, instructor, email, enroll_data)
+ self.assertEqual(len(self.book.courses), 2, "add_course: Number of courses is incorrect.")
+ self.assertEqual(len(self.book.sections), 2, "add_course: Number of sections is incorrect.")
+ self.assertEqual(self.book.sec_size[-1], 1, "add_course: Incorrect sec_size array number.")
+ self.assertEqual(self.book.total_enroll, 100, "add_course: Total enrollment count is incorrect.")
+
+ # add_section method
+ def test_add_section(self):
+ course_name = "Example Course"
+ section_num = 999
+ instructor = "Test Name"
+ email = "Test Email"
+ enroll_data = [99, "D"]
+
+ self.book.add_section(course_name, section_num, instructor, email, enroll_data)
+ self.assertEqual(len(self.book.courses), 1, "add_section: Number of courses is incorrect.")
+ # adding the 0 index as book.sections is a list of lists of sections, with the index
+ # corresponding to the index of the course index
+ self.assertEqual(len(self.book.sections[0]), 2, "add_section: Number of sections is incorrect.")
+ self.assertEqual(self.book.sec_size[-1], 2, "add_section: Incorrect sec_size array number.")
+ self.assertEqual(self.book.total_enroll, 100, "add_section: Total enrollment count is incorrect.")
+
+ # add_isbn method
+ def test_add_isbn(self):
+ isbn = 2
+ self.book.add_isbn(isbn)
+ self.assertEqual(len(self.book.isbns), 2, "add_isbn: Number of ISBNs is incorrect.")
+ self.assertIn(2, self.book.isbns, "add_isbn: Missing ISBN number.")
+
+ # add_enroll method
+ def test_add_enroll(self):
+ enroll_data = [99, "D"]
+ self.book.add_enroll(enroll_data[1], enroll_data[0])
+ self.assertEqual(self.book.total_enroll, 100, "add_enroll: Total enrollment count is incorrect.")
+ self.assertIn("Ecampus", self.book.campuses, "add_enroll: Missing one of the correct campuses.")
+
+ # add_required method
+ def test_add_required(self):
+ self.book.add_required()
+ self.assertEqual("Required", self.book.requirement, "add_required: Value did not update properly.")
+
+ # End Book Class testing
+
+ # get_max_index testing
+ def test_max_index(self):
+ # setup
+ for num in range(1, 5):
+ course_name = f"Test Course {num}"
+ base_section = 1
+ base_enroll = [99, "D"]
+ instructor = "Test Name"
+ email = "Test Email"
+ self.book.add_course(course_name, base_section, instructor, email, base_enroll)
+ for sec in range(1, num):
+ sec_num = base_section + sec
+ sec_enroll = [sec + num, "C"]
+ self.book.add_section(course_name, sec_num, instructor, email, sec_enroll)
+
+ max_list = classes.get_max_index(self.book.sec_size)
+ # it finds the first index first, meaning the matching sizes are done by lowest index
+ expected = [4, 3, 2, 0, 1]
+ self.assertEqual(expected, max_list, "get_max_index: Course maximum section count order does not match expected values.")
+
+ # get_max_courses testing
+ def test_max_courses(self):
+ self.generate_book_list()
+ expected_num = max(self.course_list)
+ most_courses = classes.get_max_courses(self.book_list)
+ self.assertEqual(most_courses, expected_num, "")
+
+ # get_max_sections_list testing
+ def test_max_sections_list(self):
+ self.generate_book_list()
+ expected_size = max(self.course_list)
+
+ most_courses = classes.get_max_courses(self.book_list)
+ section_size_list = classes.get_max_sections_list(self.book_list, most_courses)
+
+ max_list_size = 0
+ for sec_list in self.section_list:
+ max_list_size = max(len(sec_list), max_list_size)
+
+ expected_list = []
+ for idx in range(0, max_list_size + 1):
+ max_idx = 0
+ for sec_list in self.section_list:
+ if len(sec_list) - 1 >= idx:
+ max_idx = max(sec_list[idx], max_idx)
+ expected_list.append(max_idx)
+
+ for idx, sec_size in enumerate(section_size_list):
+ self.assertGreater(expected_size, idx)
+ self.assertEqual(sec_size, expected_list[idx])
+
+ # process_book testing
+ def test_process_book(self):
+ self.generate_book_list()
+ local_list = classes.process_book(self.book_list, self.book_info)
+ self.book_list.append(self.book)
+ for book_idx in range(0, len(local_list)):
+ self.assertEqual(local_list[book_idx], self.book_list[book_idx])
+
+ # process_courses testing
+ def test_process_courses(self):
+ simplefilter(action="ignore", category=pd.errors.PerformanceWarning)
+ simplefilter(action="ignore", category=FutureWarning)
+
+ self.generate_book_list()
+ dataframe = pd.DataFrame()
+ head_names, main_headers = util.get_sheet_headers()
+ total_cols = 0
+ for header in main_headers:
+ dataframe[f"{header}"] = []
+ total_cols += 1
+ format_headers = util.get_format_headers()
+
+ max_len = 0
+ for book in self.book_list:
+ max_len = max(len(book.courses), max_len)
+ total_cols += max_len
+
+ max_courses = classes.process_courses(self.book_list, format_headers, dataframe)
+ self.assertEqual(max_len, max_courses)
+ self.assertEqual(total_cols, len(dataframe.columns))
+
+ # process_sections testing
+ def test_process_sections(self):
+ simplefilter(action="ignore", category=pd.errors.PerformanceWarning)
+ simplefilter(action="ignore", category=FutureWarning)
+
+ self.generate_book_list()
+ dataframe = pd.DataFrame()
+ head_names, main_headers = util.get_sheet_headers()
+ total_cols = 0
+ for header in main_headers:
+ dataframe[f"{header}"] = []
+ total_cols += 1
+ format_headers = util.get_format_headers()
+
+ max_len = 0
+ for book in self.book_list:
+ max_len = max(len(book.courses), max_len)
+ total_cols += max_len
+
+ most_courses = classes.get_max_courses(self.book_list)
+ section_size_list = classes.get_max_sections_list(self.book_list, most_courses)
+ for sec_size in section_size_list:
+ total_cols += sec_size * 4
+
+ max_courses = classes.process_courses(self.book_list, format_headers, dataframe)
+ classes.process_sections(self.book_list, format_headers, max_courses, dataframe)
+
+ self.assertEqual(total_cols, len(dataframe.columns))
+
+ # process_isbns testing
+ def test_process_isbns(self):
+ simplefilter(action="ignore", category=pd.errors.PerformanceWarning)
+ simplefilter(action="ignore", category=FutureWarning)
+
+ self.generate_book_list()
+ dataframe = pd.DataFrame()
+ head_names, main_headers = util.get_sheet_headers()
+ total_cols = 0
+ for header in main_headers:
+ dataframe[f"{header}"] = []
+ total_cols += 1
+ format_headers = util.get_format_headers()
+
+ max_len = 0
+ max_isbn = 0
+ for book in self.book_list:
+ max_len = max(len(book.courses), max_len)
+ max_isbn = max(len(book.isbns), max_isbn)
+ total_cols += max_len
+
+ most_courses = classes.get_max_courses(self.book_list)
+ section_size_list = classes.get_max_sections_list(self.book_list, most_courses)
+ for sec_size in section_size_list:
+ total_cols += sec_size * 4
+
+ max_courses = classes.process_courses(self.book_list, format_headers, dataframe)
+ classes.process_sections(self.book_list, format_headers, max_courses, dataframe)
+ classes.process_isbns(self.book_list, head_names, main_headers, dataframe)
+
+ self.assertEqual(total_cols, len(dataframe.columns))
+
+ # import_data testing
+ def test_import_data(self):
+ simplefilter(action="ignore", category=pd.errors.PerformanceWarning)
+ simplefilter(action="ignore", category=FutureWarning)
+
+ self.generate_book_list()
+ dataframe = pd.DataFrame()
+ head_names, main_headers = util.get_sheet_headers()
+ for header in main_headers:
+ dataframe[f"{header}"] = []
+ dataframe[f"{header}"] = dataframe[f"{header}"].astype(object)
+ format_headers = util.get_format_headers()
+
+ max_courses = classes.process_courses(self.book_list, format_headers, dataframe)
+ classes.process_sections(self.book_list, format_headers, max_courses, dataframe)
+ classes.process_isbns(self.book_list, head_names, main_headers, dataframe)
+ dataframe = classes.import_data(self.book_list, format_headers, head_names, dataframe)
+ self.assertEqual(len(self.book_list), len(dataframe))
+
+
+class EmailsTest(unittest.TestCase):
+ """Testing the emails.py file."""
+ def setUp(self):
+ book_title = "Book Title"
+ book_author = "Book Author"
+ book_edition = "Book Edition"
+ book_year = "Book Year"
+ book_access = {
+ "Ebook": {
+ "Number": 1,
+ "Link": "ebook cdl link",
+ "CDL": True
+ },
+ "Print1": {
+ "Number": 2,
+ "Link": "print 1 link"
+ },
+ "Print2": {
+ "Number": 4,
+ "Link": "print 2 link"
+ },
+ "Audio": {
+ "Link": "audio link"
+ }
+ }
+ self.emails_book = emails.Book(book_title, book_author, book_edition, book_year, book_access)
+
+ inst_name = "Instructor Name"
+ inst_email = "Instructor Email"
+ inst_course = "Course 1"
+ inst_section = "101"
+ self.emails_instructor = emails.Instructor(inst_name, inst_email, inst_course, inst_section, self.emails_book)
+
+ # Instructor Class testing
+
+ # add_book method
+ def test_add_book(self):
+ title = "Book 2"
+ author = "Author 2"
+ edition = "Edition 2"
+ year = "Year 2"
+ access = {"Ebook": {
+ "Number": 7,
+ "Link": "ebook link",
+ "CDL": False
+ }}
+ local_book = emails.Book(title, author, edition, year, access)
+
+ case_courses = ["Course 1", "Course 1", "Course 2"]
+ case_sections = ["101", "102", "101"]
+ for idx, course in enumerate(case_courses):
+ self.emails_instructor.add_book(course, case_sections[idx], local_book)
+
+ self.assertEqual(2, len(self.emails_instructor.data))
+ self.assertEqual(2, len(self.emails_instructor.data["Course 1"]))
+ self.assertEqual(2, len(self.emails_instructor.data["Course 1"]["101"]))
+ self.assertEqual(1, len(self.emails_instructor.data["Course 1"]["102"]))
+ self.assertEqual(1, len(self.emails_instructor.data["Course 2"]))
+ self.assertEqual(1, len(self.emails_instructor.data["Course 2"]["101"]))
+
+ # could add in more tests beyond just the length checking but the data issues would
+ # be apparent when generating emails as well
+
+
+ # End Instructor Class testing
+
+ # update_excel testing
+ # this needs a setup to run sheetmaker and import other information?
+ def test_update_excel(self):
+ bookstore_dir = get_directory(["testing", "test_bookstore.csv"])
+ output_dir = get_directory(["testing", "test_output.xlsx"])
+
+ workbook = openpyxl.load_workbook(output_dir)
+ sheet_name = "Test"
+ worksheet = workbook[sheet_name]
+
+ header_dict, header_list = util.get_sheet_headers()
+
+ # generate test data
+ email_sent_list = []
+ email_ready_list = []
+ test_time = datetime.datetime.now()
+ for idx, row in enumerate(worksheet.iter_rows(min_row=2)):
+ if idx >= 151:
+ break
+ prior_check = False
+ for cell in row:
+ col = cell.column_letter
+ if worksheet[f"{col}1"].value == header_dict["ReadingList"]:
+ num = get_random(1, 2)
+ if num == 1:
+ cell.value = True
+ prior_check = True
+ email_ready_list.append(idx)
+ else:
+ cell.value = False
+ elif worksheet[f"{col}1"].value == header_dict["EmailDate"]:
+ num = get_random(1, 2)
+ if num == 1 or prior_check == False:
+ cell.value = None
+ else:
+ email_ready_list.pop()
+ email_sent_list.append(idx)
+ cell.value = test_time
+
+ workbook.save(output_dir)
+ workbook.close()
+
+ emails.update_excel(output_dir, email_ready_list, sheet_name)
+
+ final_email_list = sorted(email_ready_list + email_sent_list)
+
+ workbook = openpyxl.load_workbook(output_dir)
+ worksheet = workbook[sheet_name]
+ for idx, row in enumerate(worksheet.iter_rows(min_row=2)):
+ if idx >= 151:
+ break
+ reading_val = None
+ email_val = None
+ for cell in row:
+ col = cell.column_letter
+ if worksheet[f"{col}1"].value == header_dict["ReadingList"]:
+ reading_val = cell.value
+ elif worksheet[f"{col}1"].value == header_dict["EmailDate"]:
+ email_val = cell.value
+ if idx in final_email_list:
+ self.assertTrue(reading_val)
+ self.assertTrue(email_val)
+ else:
+ self.assertFalse(reading_val)
+ self.assertFalse(email_val)
+
+
+ # create_email_excel testing
+ def test_create_emails(self):
+ output_dir = get_directory(["testing", "test_output.xlsx"])
+
+ workbook = openpyxl.load_workbook(output_dir)
+ sheet_name = "Test"
+ worksheet = workbook[sheet_name]
+
+ header_dict, header_list = util.get_sheet_headers()
+
+ # generate test data
+ email_sent_list = []
+ email_ready_list = []
+ test_time = datetime.datetime.now()
+ for idx, row in enumerate(worksheet.iter_rows(min_row=2)):
+ if idx >= 151:
+ break
+ prior_check = False
+ for cell in row:
+ col = cell.column_letter
+ if worksheet[f"{col}1"].value == header_dict["ReadingList"]:
+ num = get_random(1, 2)
+ if num == 1:
+ cell.value = True
+ prior_check = True
+ email_ready_list.append(idx)
+ else:
+ cell.value = False
+ elif worksheet[f"{col}1"].value == header_dict["EmailDate"]:
+ num = get_random(1, 2)
+ if num == 1 or prior_check == False:
+ cell.value = None
+ else:
+ email_ready_list.pop()
+ email_sent_list.append(idx)
+ cell.value = test_time
+
+ workbook.save(output_dir)
+ workbook.close()
+ emails.create_email_excel("Test", output_dir)
+
+ final_email_list = sorted(email_ready_list + email_sent_list)
+ email_dir = get_directory(["..", "email_output.xlsx"])
+
+ email_workbook = openpyxl.load_workbook(email_dir)
+ email_worksheet = email_workbook["Email List"]
+
+ for row in email_worksheet.iter_rows(min_row=2):
+ for cell in row:
+ col = cell.column_letter
+ if email_worksheet[f"{col}1"] == "Name":
+ name_col_list = cell.value.split(" ")
+ number = int(name_col_list[2])
+ self.assertIn(number, final_email_list)
+ break
+
+ # write_to_excel testing -- this is an export function and should be fine
+
+
+class EnrollTest(unittest.TestCase):
+ """Testing the enrollment.py file."""
+ def setUp(self):
+ directory = ["testing", "test_enrollment.csv"]
+ self.enroll_dir = get_directory(directory)
+ self.instructor_dict = {"Name1 Name2": "Name@email.com"}
+
+ def check_instructor_dict(self, instr_dict, name, email):
+ if name not in instr_dict:
+ return False
+ if email not in instr_dict[name]:
+ return False
+ return True
+
+ def test_get_enrollment_without(self):
+ instr_dict, enroll_dict = enroll.get_enrollment_data(self.enroll_dir)
+ self.assertIn("MTH101", enroll_dict, "")
+ self.assertIn("1", enroll_dict["MTH101"], "")
+
+ ex_name = "LastName, FirstName"
+ ex_email = "FirstName.LastName@example.edu"
+ check = self.check_instructor_dict(instr_dict, ex_name, ex_email)
+ self.assertTrue(check, "")
+
+ def test_get_enrollment_with(self):
+ instr_dict, enroll_dict = enroll.get_enrollment_data(self.enroll_dir, self.instructor_dict)
+ self.assertIn("MTH101", enroll_dict, "")
+ self.assertIn("1", enroll_dict["MTH101"], "")
+
+ ex_name = "LastName, FirstName"
+ ex_email = "FirstName.LastName@example.edu"
+ check = self.check_instructor_dict(instr_dict, ex_name, ex_email)
+ self.assertTrue(check, "")
+
+ for name in self.instructor_dict:
+ email = self.instructor_dict[name]
+ check = self.check_instructor_dict(self.instructor_dict, name, email)
+ self.assertTrue(check, "")
+
+
+class OutlookTest(unittest.TestCase):
+ """Testing the grabber.py file."""
+ # every function aside from the importer and exporter deal with selenium webdrivers
+
+
+class GUITest(unittest.TestCase):
+ """Testing the gui.py file."""
+ # automated gui testing falls under a similar circumstance to testing selenium
+ # there are ways and methods to do so, but for now, not enough time
+
+
+class HelperGUITest(unittest.TestCase):
+ """Testing the helpergui.py file."""
+ # see above GUI notes, automated testing gui not within scope as of right now
+ # additionally, it is easy to manually test and verify results
+
+
+class ModesTest(unittest.TestCase):
+ """Testing the modes.py file."""
+ # functions to primarily test here would be the excel sheet updaters
+
+ # enrollment_update testing
+
+ # analytics_update testing
+
+ # emails_update testing
+
+
+class OutputTest(unittest.TestCase):
+ """Testing the output.py file."""
+ # this file hosts a single function for the pure reason of handling
+ # formatting and finishing out the export for an excel file
+ # it should be fine to leave out of testing for now
+
+
+class SheetTest(unittest.TestCase):
+ """Testing the sheetmaker.py file."""
+ # only testing the version of the function with no gui, no cli interaction
+
+ # make_excel_sheet testing
+
+
+class UtilTest(unittest.TestCase):
+ """Testing the utilities.py file."""
+
+ # get_int testing
+ def test_get_int(self):
+ values = [" 12 4 5 ", "93478", None, "", " ", 65.435343, 109]
+ expected = [1245, 93478, None, None, None, 65, 109]
+ for idx, value in enumerate(values):
+ result = util.get_int(value)
+ self.assertEqual(expected[idx], result, f"get_int: result {result} does not match {expected[idx]}.")
+
+ # get_clean testing
+ def test_get_clean(self):
+ test_cleaner = ["Example", "WORDS", "something"]
+ test_names = [
+ "a book that has examples and wOrdS SOMEthing to do with an example e-reading",
+ "what other EXAMPLE case isn't here",
+ "'what about words appearing' or something elsewhere"
+ ]
+ expected = [
+ "A Book That Has S And To Do With An E-Reading",
+ "What Other Case Isn't Here",
+ "'What About Appearing' Or Elsewhere"
+ ]
+ for idx, name in enumerate(test_names):
+ result = util.get_clean(test_cleaner, name)
+ self.assertEqual(expected[idx], result)
+
+ # get_state testing
+ def test_get_state(self):
+ cases = ["true", "hmmm", "anything", "false", "FALSE", "False", "True"]
+ expected = [True, True, True, True, True, False, True]
+ for idx, case in enumerate(cases):
+ result = bool(util.get_state(case))
+ self.assertEqual(expected[idx], result)
+
+ # get_directory testing
+ def test_get_directory(self):
+ test_config = configparser.ConfigParser()
+ test_config.read("helpers/testing/test_config.ini")
+ test_config = test_config["test"]
+ result = util.get_directory("Test", test_config)
+ expected = get_directory(["testing", "test_config.csv"])
+ self.assertEqual(expected, result, f"get_directory: result {result} does not match {expected}.")
+
+ # get_filepath testing
+ def test_get_filepath(self):
+ cases = ["testing"]
+ expected = [get_directory(["..", "testing"])]
+ false_expected = [get_directory(["testing"])]
+ for idx, case in enumerate(cases):
+ result = util.get_filepath(case)
+ false_result = util.get_filepath(case, home=False)
+ self.assertEqual(expected[idx], result)
+ self.assertEqual(false_expected[idx], false_result)
+
+ # get_letter testing
+ def test_get_letter(self):
+ cases = [1, 27, 4, 53, 55]
+ expected = ["A", "AA", "D", "BA", "BC"]
+ for idx, case in enumerate(cases):
+ result = util.get_letter(case)
+ self.assertEqual(expected[idx], result)
+
+ # get_edition_string testing
+ def test_get_edition_string(self):
+ cases = [None, "", "5", 29, 10, "1", 11, 21, 22, 13]
+ expected = [None, None, "5th", "29th", "10th", "1st", "11th", "21st", "22nd", "13th"]
+ for idx, case in enumerate(cases):
+ result = util.get_edition_string(case)
+ self.assertEqual(expected[idx], result)
+
+ # get_format_headers just grabs a set of information
+ # as it is supposed to be the central point of modification for those strings
+
+ # get_replace_header testing
+ def test_get_replace_header(self):
+ format_headers = util.get_format_headers()
+ cases = [["TestNum", "TestSection", "TestEmailInfo"]]
+ for idx, case in enumerate(cases):
+ result = util.get_replace_header(format_headers[2], case[0], case[1], case[2])
+ for val in case:
+ self.assertIn(val, result)
+
+ # get_split_course testing
+ def test_get_split_course(self):
+ cases = ["MTH251", "BIO305", "WR221Z"]
+ expected = [["MTH", "251"], ["BIO", "305"], ["WR", "221Z"]]
+ for idx, case in enumerate(cases):
+ result = util.get_split_course(case)
+ self.assertEqual(expected[idx], result)
+
+ # skipping get_input and get_enabled as it is an input oriented function
+
+ # get_sheet_headers, get_config_headers, and get_string_cleaners
+ # are all also is information grabbing functions
+
+ # get_row_info testing
+ def test_get_row_info(self):
+ # an implementation can be made for row data to be tested, but there is
+ # only one spot this gets used in, so it is not relevant to time to get done
+ pass
+
+ # get_campus testing
+ def test_get_campus(self):
+ # this is breakable via any string starting with D and then
+ # filling the rest with non-existent values, but this usually does
+ # not occur in the natural data this script will be managing
+ cases = ["C", "D", "Z", "L", "N", "B", "H", "PDX", "DC", "23432", "CR"]
+ expected = ["Corvallis", "Ecampus", "International", "LaGrande", "Newport", "Cascades", "Portland", "Portland", "Ecampus", None, None]
+ for idx, case in enumerate(cases):
+ result = util.get_campus(case)
+ self.assertEqual(expected[idx], result)
+
+ # skipping set_col_format as it is likely to fluctuate and change
+
+# some helper functions to carry out testing various features
+
+def get_directory(folder_list):
+ curr_dir = os.path.dirname(__file__)
+ for file in folder_list:
+ curr_dir = os.path.join(curr_dir, file)
+ return curr_dir
+
+
+def get_random(low, high):
+ return random.randint(low, high)
diff --git a/helpers/testing/test_analytics.csv b/helpers/testing/test_analytics.csv
new file mode 100644
index 0000000..c9e55ff
--- /dev/null
+++ b/helpers/testing/test_analytics.csv
@@ -0,0 +1 @@
+978111111, "{'11111': {'Types': ['Physical'], 'Copies': [3], 'Users': [0], 'CDL': [False], 'Link': 'https://www.example1.com', 'Year': '2022', 'Location': 'Valley Reserves'}, '11112': {'Types': ['Electronic'], 'Copies': [0], 'Users': ['unlimited'], 'CDL': [False], 'Link': 'https://www.example2.com', 'Platform': 'EBSCO (Ebooks-Purchased)', 'Year': '2022'}}"
\ No newline at end of file
diff --git a/helpers/testing/test_bookstore.csv b/helpers/testing/test_bookstore.csv
new file mode 100644
index 0000000..84d1e08
--- /dev/null
+++ b/helpers/testing/test_bookstore.csv
@@ -0,0 +1 @@
+2022-Summer,MTH : Mathematics,101,001,"LastName, FirstName",Intro to Math Textbook,11,Textbook Author,978-1-11-111,Textbook Publisher,Optional,1,Some Comment,2/2/2022 2:22:22 PM
\ No newline at end of file
diff --git a/helpers/testing/test_config.ini b/helpers/testing/test_config.ini
new file mode 100644
index 0000000..4cb09a2
--- /dev/null
+++ b/helpers/testing/test_config.ini
@@ -0,0 +1,3 @@
+[test]
+TestDir = testing
+TestFile = test_config.csv
\ No newline at end of file
diff --git a/helpers/testing/test_emails.csv b/helpers/testing/test_emails.csv
new file mode 100644
index 0000000..ab44d03
--- /dev/null
+++ b/helpers/testing/test_emails.csv
@@ -0,0 +1 @@
+"LastName, FirstName", LastName.FirstName@example.edu
\ No newline at end of file
diff --git a/helpers/testing/test_enrollment.csv b/helpers/testing/test_enrollment.csv
new file mode 100644
index 0000000..5d71893
--- /dev/null
+++ b/helpers/testing/test_enrollment.csv
@@ -0,0 +1,5 @@
+AcademicYear,AcademicPeriod,AS_OF,CourseLevelParameter,CampusParameter
+Academic Year 2021-22,Summer 2022,2/22/2022,"100, 200, 300, 400, 500, 600, 700, 800","C- Oregon State - Corvallis, DZ-Ecampus-Distance Ed Internatl, DI-Ecampus-Distance Education-LD, DB-Ecampus-Distance Education-UD, L-LaGrande/EOU, N-Newport/HMSC, B-Oregon State - Cascades, PDX-Oregon State - Portland, H-Portland/OHSU"
+
+COLLEGE,COLLEGE_DESC,DEPARTMENT_DESC,ACADEMIC_PERIOD,SUBJECT,SUBJECT_DESC,COURSE_IDENTIFICATION,OFFERING_NUMBER,COURSE_REFERENCE_NUMBER,TITLE_SHORT_DESC,CAMPUS,MAX_CREDITS,SCHEDULE_DESC,MAXIMUM_ENROLLMENT,ACTUAL_ENROLLMENT,SEATS_AVAILABLE,SECTION_CROSS_LIST,CROSS_LIST_MAXIMUM_ENROLLMENT,CROSS_LIST_ENROLLMENT,CROSS_LIST_SEATS_AVAILABLE,GENERATED_CREDITS,PRIMARY_INSTRUCTOR,INTERNET_ADDRESS,PERCENTAGE_OF_RESPONSIBILITY,INSTRUCTOR_COUNT,MEETING_TIMES,LOCATION
+1,College Desc Example,Department Desc Example,202300 - Summer 2022,MTH,Mathematics,MTH101,001,11111,Intro to Math,C,3,In-Person,100,1,99,,,,,300,"LastName, FirstName",FirstName.LastName@example.edu,100,1,,
\ No newline at end of file
diff --git a/helpers/testing/test_output.xlsx b/helpers/testing/test_output.xlsx
new file mode 100644
index 0000000..523f5e5
Binary files /dev/null and b/helpers/testing/test_output.xlsx differ
diff --git a/helpers/utilities.py b/helpers/utilities.py
index 163e5f9..e7ee531 100644
--- a/helpers/utilities.py
+++ b/helpers/utilities.py
@@ -12,6 +12,7 @@ def get_int(value):
if type(value) is str and not pd.isna(value):
if not value.strip() == "":
if not any(char.isalpha() for char in value):
+ value = value.replace(" ", "")
output = int(value)
elif type(value) is float and not pd.isna(value):
output = int(value)
@@ -53,7 +54,32 @@ def get_clean(cleaner, name):
temp_list[kdx + 1] = name[kdx + 1].lower()
name = "".join(temp_list)
- return f"{name}"
+ temp = name
+ new_name = ""
+ first = True
+ while new_name is not temp:
+ if first:
+ first = False
+ else:
+ temp = new_name
+ new_name = temp.strip()
+ new_name = new_name.replace(" ", "")
+ new_name = new_name.replace(" ", " ")
+ new_name = new_name.replace("\t", "")
+ new_name = new_name.replace("\n", "")
+
+ prior = ""
+ for jdx, letter in enumerate(new_name):
+ if jdx == 0:
+ prior = letter
+ continue
+ if prior == " ":
+ new_name = list(new_name)
+ new_name[jdx] = new_name[jdx].upper()
+ new_name = "".join(new_name)
+ prior = letter
+
+ return f"{new_name}"
def get_state(config):
@@ -113,11 +139,34 @@ def get_edition_string(num):
if edition_num is not None:
if edition_num[-1] == "1":
- edition_num += "st"
+ # special case for 11, 12, 13
+ if len(edition_num) > 1:
+ if edition_num[-2] == "1":
+ edition_num += "th"
+ else:
+ edition_num += "st"
+
+ else:
+ edition_num += "st"
+
elif edition_num[-1] == "2":
- edition_num += "nd"
+ if len(edition_num) > 1:
+ if edition_num[-2] == "1":
+ edition_num += "th"
+ else:
+ edition_num += "nd"
+ else:
+ edition_num += "nd"
+
elif edition_num[-1] == "3":
- edition_num += "rd"
+ if len(edition_num) > 1:
+ if edition_num[-2] == "1":
+ edition_num += "th"
+ else:
+ edition_num += "rd"
+ else:
+ edition_num += "rd"
+
elif edition_num[-1] in th_list:
edition_num += "th"
@@ -222,7 +271,7 @@ def get_sheet_headers():
def get_config_headers():
- """"""
+ """Gets the config file reading for the headers.ini file."""
config = configparser.ConfigParser()
config.read("helpers/ini/headers.ini")
return config
diff --git a/main.py b/main.py
index b95093c..5cb9079 100644
--- a/main.py
+++ b/main.py
@@ -5,14 +5,9 @@
from helpers.gui import start_app
from helpers.sheetmaker import make_excel_sheet
-# General TODO:
-# finish isbn updating from the bookstore (i.e. isbn differs from what we own)
-# fully finish documentation.md
-# add images to readme
-
# TODO
# This is a list of improvements that I am aware may be able to be added
-# However, due to time constraints or lack of information are left as is
+# However, due to time constraints or lack of information are left for later
# - try / except blocks around data entry code
# things like analytics.py could use more error checking to prevent
@@ -30,6 +25,14 @@
# not all of these are serious needs, requirements, or fixes
# as much as they are probably spots i have left notes of
# what COULD be done or what hasn't been fully tested
+# - adding testcases for selenium functions using hidden github values
+# could still login using headless functionality and secret key values
+# - adding in cost comparison that works without querying the site too many times
+# - improving documentation, comments, and overall code structure
+# - cross referencing our own ISBN values from the bookstore ISBN values
+# sometimes we own a different book than what the bookstore has per class
+# but it is the same book, in which case this could use an automatic cross
+# referencing function to automatically pull those values via titles
config = configparser.ConfigParser()
config.read("config.ini")
diff --git a/runtest.py b/runtest.py
new file mode 100644
index 0000000..619d676
--- /dev/null
+++ b/runtest.py
@@ -0,0 +1,9 @@
+from helpers.test import *
+import unittest
+
+# this file just ensures that test.py is run within the correct scope of
+# the helpers folder to allow all other files to properly reference
+# their dependencies
+
+if __name__ == '__main__':
+ unittest.main()