siglatools.bin package#

Submodules#

siglatools.bin.get_next_uv_dates module#

This script will get deployed in the bin directory of the users’ virtualenv when the parent module is installed using pip.

class siglatools.bin.get_next_uv_dates.Args[source]#: Bases: Namespace

class siglatools.bin.get_next_uv_dates.CheckedNextUVDate(status: NextUVDateStatus, next_uv_date_data: NextUVDateData)[source]#

Bases: tuple

The status of a next uv date after checking.

Attributes:

status: NextUVDateStatus: The status of a next uv date after checking.
next_uv_date_data: NextUVDateData: The next uv date and its context. See NextUVDateData class.

Create new instance of CheckedNextUVDate(status, next_uv_date_data)

next_uv_date_data: NextUVDateData#

status: NextUVDateStatus#

class siglatools.bin.get_next_uv_dates.NextUVDateData(spreadsheet_title: str, sheet_title: str, column_name: str, row_index: int, next_uv_date: date)[source]#

Bases: tuple

A next update and verify date and its context

Attributes:

spreadsheet_title: str: The title of the spreadsheet the contains the next uv date.
sheet_title: str: The title of the sheet that contains the next uv date.
column_name: str: The column location of the next uv date in the sheet.
row_index: int: The row location of the next uv date in the sheet.
next_uv_date: str: The next uv date.

Create new instance of NextUVDateData(spreadsheet_title, sheet_title, column_name, row_index, next_uv_date)

column_name: str#

next_uv_date: date#

row_index: int#

sheet_title: str#

spreadsheet_title: str#

class siglatools.bin.get_next_uv_dates.NextUVDateStatus[source]#

Bases: object

Possible status of a next uv date.

incorrect_date_format = 'Incorrect date format'#

irrelevant = 'Irrelevant'#

requires_uv = 'Requires update and verify'#

siglatools.bin.get_next_uv_dates.get_next_uv_dates(master_spreadsheet_id: str, google_api_credentials_path: str, start_date: date, end_date: date)[source]#

Get next update and verify dates or uv dates that falls within the date range.

Parameters:

master_spreadsheet_id (str) – The master spreadsheet id
google_api_credentials_path (str) – The path to Google API credentials file needed to read Google Sheets.
start_date (date) – The start date.
end_date (date) – The end date.

siglatools.bin.get_next_uv_dates.main()[source]#

siglatools.bin.load_spreadsheets module#

This script will get deployed in the bin directory of the users’ virtualenv when the parent module is installed using pip.

class siglatools.bin.load_spreadsheets.Args[source]#: Bases: Namespace

siglatools.bin.load_spreadsheets.load_spreadsheets(spreadsheet_ids: List[str], db_connection_url: str, google_api_credentials_path: str)[source]#

Load spreadsheets to the database.

Parameters:

spreadsheet_ids (List[str]) – The list of spreadsheet ids.
db_connection_url (str) – The DB’s connection url str.
google_api_credentials_path (str) – The path to Google API credentials file needed to read Google Sheets.

siglatools.bin.load_spreadsheets.main()[source]#

siglatools.bin.run_external_link_checker module#

This script will get deployed in the bin directory of the users’ virtualenv when the parent module is installed using pip.

class siglatools.bin.run_external_link_checker.Args[source]#: Bases: Namespace

class siglatools.bin.run_external_link_checker.CheckedURL(has_error: bool, url_data: URLData, msg: Optional[str] = None)[source]#

Bases: tuple

The status of an URL after checking.

Attributes:

has_error: bool: Whether the URL has an error.
url_data: URLData: The URL and its context. See URLData class.
msg: Optional[str] = None: The status of the URL after checking. None if has_error is False.

Create new instance of CheckedURL(has_error, url_data, msg)

has_error: bool#

msg: Optional[str]#

url_data: URLData#

class siglatools.bin.run_external_link_checker.GoogleSheetCell(spreadsheet_title: str, sheet_title: str, row_index: int, col_index: int, url: Optional[str] = None, msg: Optional[str] = None)[source]#

Bases: tuple

A GoogleSheet cell.

Attributes:

spreadsheet_title: str: The title of the spreadsheet the contains the URL.
sheet_title: str: The title of the sheet that contains the URL.
row_index: int: The row index of the cell.
col_index: int: The col index of the cell.
url: Optional[str] = None: The url in the cell.
msg: Optional[str] = None: The status of url.

Create new instance of GoogleSheetCell(spreadsheet_title, sheet_title, row_index, col_index, url, msg)

col_index: int#

msg: Optional[str]#

row_index: int#

sheet_title: str#

spreadsheet_title: str#

url: Optional[str]#

class siglatools.bin.run_external_link_checker.URLData(cells: List[GoogleSheetCell], url: str)[source]#

Bases: tuple

The URL and its context.

Attributes:

cells: List[GoogleSheetCell]: The list of cells that has the URL.
url: str: The URL.

Create new instance of URLData(cells, url)

add_cell(cell: GoogleSheetCell)[source]#: Add a cell to the list of cells.

cells: List[GoogleSheetCell]#

get_key() → str[source]#: Get the key.

url: str#

siglatools.bin.run_external_link_checker.main()[source]#

siglatools.bin.run_external_link_checker.run_external_link_checker(google_api_credentials_path: str, master_spreadsheet_id: Optional[str] = None, spreadsheet_ids_str: Optional[str] = None)[source]#

Run the the external link checker. If a list of spreadsheet ids are provided, run the external link checker against the list of spreadsheet ids, instead of the spreadsheet ids gathered from the master spreadsheet.

Parameters:

master_spreadsheet_id (str) – The master spreadsheet id.
google_api_credentials_path (str) – The path to Google API credentials file needed to read Google Sheets.
spreadsheet_ids_str (Optional[str]) – The list spreadsheet ids, delimited by comma.

siglatools.bin.run_qa_test module#

This script will get deployed in the bin directory of the users’ virtualenv when the parent module is installed using pip.

class siglatools.bin.run_qa_test.Args[source]#: Bases: Namespace

class siglatools.bin.run_qa_test.Comparison(spreadsheet_title: str, sheet_title: str, name: str, data_comparisons: List[ObjectComparison])[source]#

Bases: tuple

A group of ObjectComparison.

Attributes:

spreadsheet_title: str: The spreadsheet source of the GoogleSheet data.
sheet_title: str: The sheet source of the GoogleSheet data.
name: str: The name of the group.
data_comparisons: List[ObjectComparison]: The group of ObjectComparison.

Create new instance of Comparison(spreadsheet_title, sheet_title, name, data_comparisons)

data_comparisons: List[ObjectComparison]#

get_filename() → str[source]#: Get the file name.

get_gs_source() → str[source]#: Get the GoogleSheet source.

get_name() → str[source]#: Return the name of the comparison with / replaced with |.

has_error() → bool[source]#: Does any of the group of ObjectCOmparison has a mismatch?

name: str#

sheet_title: str#

spreadsheet_title: str#

write() → str[source]#

Write the mistmatch data_comparisons to a file.

Returns:: filename – The filename.
Return type:: str

class siglatools.bin.run_qa_test.Datasource[source]#

Bases: object

database = 'Database'#

googlesheet = 'GoogleSheet'#

class siglatools.bin.run_qa_test.FieldComparison(field: str, comparison_type: str, actual: Tuple[str, Any], expected: Tuple[str, Any])[source]#

Bases: tuple

A comparison of two values for a field.

Attributes:

field: str: The field name.
comparison_type: str: The field comparison type.
actual: Tuple[str, Any]: A value and its source.
expected: Tuple[str, Any]: A value and its source.

Create new instance of FieldComparison(field, comparison_type, actual, expected)

actual: Tuple[str, Any]#

comparison_type: str#

expected: Tuple[str, Any]#

field: str#

get_error_msg() → str[source]#: Get error msg.

get_str_rep() → str[source]#: Get str of the two values.

has_error() → bool[source]#: Does the two values mismatch?

class siglatools.bin.run_qa_test.FieldComparisonType[source]#

Bases: object

data = 'data'#

meta = 'meta'#

class siglatools.bin.run_qa_test.GsInstitution(spreadsheet_id: str, spreadsheet_title: str, sheet_id: str, sheet_title: str, data: Any)[source]#

Bases: tuple

An institution from GoogleSheet.

Attributes:

spreadsheet_id: str: The spreadsheet id.
spreadsheet_title: str: The spreadsheet title.
sheet_id: str: The sheet id.
sheet_title: str: The sheet title.
data: Any: The institution object and its variables.

Create new instance of GsInstitution(spreadsheet_id, spreadsheet_title, sheet_id, sheet_title, data)

data: Any#

get_key() → str[source]#: Get the key of the institution.

sheet_id: str#

sheet_title: str#

spreadsheet_id: str#

spreadsheet_title: str#

class siglatools.bin.run_qa_test.ObjectComparison(name: str, field_comparisons: List[FieldComparison])[source]#

Bases: tuple

A group of FieldComparison.

Attributes:

name: str: The name of the group.
field_comparisons: List[FieldComparison]: The group of FieldComparison.

Create new instance of ObjectComparison(name, field_comparisons)

field_comparisons: List[FieldComparison]#

get_error_msgs() → List[str][source]#: Get a list of msg for the mistmatch FieldComparison.

has_error() → bool[source]#: Does any of the group of FieldComparison have a mismatch?

name: str#

siglatools.bin.run_qa_test.main()[source]#

siglatools.bin.run_qa_test.run_qa_test(db_connection_url: str, google_api_credentials_path: str, master_spreadsheet_id: Optional[str] = None, spreadsheet_ids_str: Optional[str] = None)[source]#

Run QA test

Parameters:

master_spreadsheet_id (str) – The master spreadsheet id.
db_connection_url (str) – The DB’s connection url str.
google_api_credentials_path (str) – The path to Google API credentials file needed to read Google Sheets.
spreadsheet_ids_str (Optional[str] = None) – The list of spreadsheet ids.

siglatools.bin.run_sigla_pipeline module#

This script will get deployed in the bin directory of the users’ virtualenv when the parent module is installed using pip.

class siglatools.bin.run_sigla_pipeline.Args[source]#: Bases: Namespace

siglatools.bin.run_sigla_pipeline.main()[source]#

siglatools.bin.run_sigla_pipeline.run_sigla_pipeline(master_spreadsheet_id: str, google_api_credentials_path: str, db_connection_url: str)[source]#

Run the SIGLA ETL pipeline

Parameters:

master_spreadsheet_id – The master spreadsheet id.
google_api_credentials_path (str) – The path to Google API credentials file needed to read Google Sheets.
db_connection_url (str) – The DB’s connection url str.

Module contents#

Bin scripts package for siglatools.