
History:

Klusterbox Version 1.0 
	Introduced a new way to investigate improper mandating violations. Version 1.0 incorporated the core functionality of klusterbox. 

Version 2.0 
	Introduced the Auto Improper Mandate Finder, later called the Auto Data Entry. This feature reads a properly formatted employee everything report (the .csv file type) and will build a carrier list as well as input carrier clock rings with minimal input from a steward. The Auto Over Max Finder will also read a properly formatted employee everything report and generate a report showing violations of the 12/60 hour rule laid out in article 8 of the jcam. These features take task which once took hours and completes them in minutes. Version 2.3 introduced Informal C, a feature which helped users to track compliance for grievance settlements. 

Version 3.0 
	Introduced the PDF converter which translates TACS Employee Everything Reports into a .csv format which can be read by Klusterbox. A PDF Splitter is also included for breaking Employee Everything Reports into manageable weekly sections.
 
Version 3.004 (7/21/2020) 
	The Over Max spreadsheet was added. Users were allowed to enter leave types/times into the database. The Auto Data Entry program also automatically found and entered the leave type/times into the database. 

Version 3.005 (11/01/2020) 
	Repairs were made to the pdf converter providing error warnings for troublesome input and reducing the risk of crashes. The pdf reader will make a second attempt with new parameters if the fist attempt fails. The overmax spreadsheet generator was updated to fix undercounting violations. Archives were renamed from "Libraries" on the menu. Formatting and option updates were made to informal c adding "verified" and "incomplete" as options for documentation status. 

Version 3.006 (12/21/2020) 
	More fixes were made to the pdf converter and reports for carrier routes and ns days were added. 

Version 3.008 (02/01/2021) 
	The first mac compatible version was released. This entailed creating variable widget lengths depending on the operating system. Also some of the colors were altered in the gui dependent on the operating system. Some changes were made to the spreadsheet formulas to make them compatible with Apple Numbers. Since the mac version can not recognize relative paths, I eliminated the kb_sub folder for the mac version and placed necessary images in the app bundle (applications/klusterbox.app/contents/resources) and I placed the database in a hidden folder in the documents directory (user/ ~ / documents/.klusterbox). The documents archive was placed in a visible folder which is created by klusterbox (user/ ~ / documents/klusterbox/).

Version 4.0 (02/11/2021) 
	This is the first version with an installer for windows. The NSIS installer is used to create a Setup_klusterbox.exe which installs the app package in the Program Files (x86) directory. Like the the mac OS app version, I put the database in a hidden folder in the documents folder (user/ ~ / documents/.klusterbox). Also, The archive is similarly placed in a folder of the documents folder (user/ ~ / documents/klusterbox/) instead of in the kb_sub folder. Needed images are kept in the app bundle, eliminating the need for the kb_sub folder. The klusterbox.exe will still be made available for windows users as a legacy version. 

Version 4.001 
	A database maintenance function was added, allowing users to clear out targeted records in the database or destroy/rebuild the database. A screen was added allowing users to purge carrier records. Also a delete button was added to the 'edit carrier' screen allows the user to purge the carrier's records.  A review of the source code was made to correct syntax to best practices. The About Klusterbox function was modified to allow for viewing of project documentation. There was some debugging in the auto data entry error checking. Five digit route numbers are allowed and regulated. PTFs are added as a list status. Upgrades were made to the GUI, reports and spreadsheets to allow for five digit route numbers and ptfs. Maintenance on the spreadsheet function code. The clock rings report and Carrier by List report were added. The Carrier Status History Report was added as a screen where users can select from a list of carriers or access it though the edit carrier function. Some debugging to the Over Max Spreadsheet was done so that values for Sunday were added to the totals. Also, a bug was fixed which allowed the same carrier to appear on the Over Max Spreadsheet twice when generated by Klusterbox. "Out of station" is no longer accessible via the station option menu in the main screen, now it is accessed via the Basic menu bar. The menu bar was re-organized. Some improvements were made to the pdf converter. 

Version 4.002 (06/28/2021)
	SpeedSheets were added which allow a user to generate a speedsheet which they can use to update carrier and clock ring information. Support for this was also built: a configuration window to adjust settings, a verification protocol, a report and database connectivity. The interface for inputting the investigation range on the main page was modified. Users can choose between three modes. 
	I did a lot of work overhauling code to an object oriented model - using classes and methods. The following were overhauled: Report Window, About Klusterbox, Start Up, Spreadsheet Configuration Window, Enter Rings Window, Main Frame and Database Setup. The program was fragmented into seven modules to reduce the size of the main python file. 

Version 4.003 (08/17/2021) 
	OTDL Equitability Spreadsheet was added allowing users to track overtime how much overtime and overtime opportunities OTDL carriers had been given over a quarter. A screen for OT Preference was added, allowing the user to change an OTDL carrier's preference between 12 or 10 hours. Also the user can select to "track" a carriers overtime, if the carrier have opted to get off the list, but inequities might still exist. Users can generate a report of carrier status changes for the quarter.
	Also a Refusals screen was added to allow users to enter, update, delete refusals for a carrier in a given quarter. The option menu for OTDL carrier codes on the enter rings screen, as well as in SpeedSheets, was modified to allow "NS Day" as this can be effectively used in the calculations for overtime. Two tables were added to the database for support - otdl_preference and refusals. The Spreadsheet Settings screen was modified with a minimum row option for OTDL Equitability and a choice between calculating overtime as straight overtime or proper overtime. 

Version 4.004 (09/02/2021) 
	This is a hot-fix for one formula on the overmax spreadsheet, the pdf converter which was crashing due to blank routes on temporary assignments and a revision to formulas on the OTDL Equitability Spreadsheet which were not working in Excel and Mac Numbers. 

Version 4.005 (09/20/2021) 
	The Overtime Distribution Spreadsheet was added. This tracks the overtime used for carriers. The user can select otdl, wal, no list, aux or ptf carriers. Also the a spreadsheet can be generated for a week or a quarter. This tool allows you to compare non-otdl carriers' overtime to each other as it is sometimes an issue that management gives more overtime to certain carriers who feel aggrieved that they are being given an unfair share of the overtime. 

Version 4.006 (01/09/2022) 
	The kbcsv_repair module was added so that csv files could be fixed if they were not properly formatted. Also the code for the Auto Data Entry was overhauled making it easier to read and edit. This will make the Auto Data Entry more reliable and stable.

Version 5.0 (03/10/2022) 
	Code was restructured to create two new modules: kbenterrings.py and kbinformalc.py. The two modules create new windows and which run independent of the main window. Mostly I wanted users to not lose their place on the carrier list when when entering rings. Also, I wanted to reduce the size of the the code on the main module - hence breaking off the informal c module. A new module - fixes - was added to allow fixes specific to new versions being installed. The fixes for 5.0 remove null values in the rings 3 table and remove all uppercase names in the database and changes them to all lowercase. 
	I created an improper mandates N0.4 spreadsheet. This is a spreadsheet provided by the branch 47 of the NALC. Klusterbox will generate the spreadsheet and fill in values with data from the database. A screen for configurations is included in the management menu.
	The option of entering begin tour and end tour were added to the rings table. Data entry modalities of enter rings screen, speedsheets and auto data entry where updated to allow the user to enter, edit and record begin tour and end tour. An auto detect for end tour was added to enter rings screen and speedsheets to make for faster data entry. The improper mandates number 4 spreadsheet was added to allow viewing of begin tour and end tour output. 
	Dispatch of Value functionality was added. A screen in the management menu allows users to keep track of and update DOV records. A default value is set if the user does not set DOV. The improper mandates number 4 spreadsheet is the only output the utilitizes DOV so far. 

Version 5.01 (04/17/2022)
	The Off Bid Finder Spreadsheet was created to allow stewards to find cases were carriers do not work on their routes or do not get 8 hours of work on their routes. A Spreadsheet setting is available to determine level of off bid work constitutes a pivot vs being worked off bid improperly as well as determining if output should produce separate pages for each carrier or all carriers should be listed without page breaks. 
	A Speedsheet settings was added "Show Full Report" which allows the user to limit the output of the report for checking/inputing the Speedsheet. True will yield only the errors. The Speedsheets were also modified so that the fields for clock ring times are formatted as with two decimal places when appropriate. 
	The Automatic Data Entry will back fill carrier's TACS names if the run and the names are not recorded (as in the case where the steward was using Speedsheets prior). The names are also sorted by kb name as opposed to TACS name in the name index. 

Version 5.02 (04/30/2022)
	This is a fix for two bugs. 1. A bug in the enter rings screen when the investigation range was set to day which would not input data into the database when the Apply/Submit button was pressed. 2. A bug in Speedsheets which would not input the very last row of any speedsheet. Both bugs were fixed with 30 lines of code added or altered. 

Version 5.03 (05/30/2022)
	In this version I overhauled and improved the pdf converter. The PDF Converter will now convert employee everything reports containing more than one week and output multiple csv files when needed. Prior to the overhaul, the code was mostly one 1000 line long function. I broke up the code into classes and methods and conformed to PEP 8 standards. I implemented error handling to keep the program from crashing with try/except statements. I also switched my email address to tomweeks@klusterbox.com

Version 5.04 (07/28/2022)
	This is a fix for a error which causes Klusterbox to crash if a certain condition is present when the  Pdf Converter is run (a variable was improperly named). 
	I added "Open Database Location" and "Back up your database" functionality to the Database Maintenance Tools. This will make it easier for users to open/ save/ move the database. 


Version 5.05 (09/03/2022)
	This version is mostly small technical fixes. There were fixes to the Automatic Data Entry, the module that creates the proxy csv files and SpeedSheets. 
	Amended the Klusterbox.py> AutoDataEnty> AutoSkimmer> fix_carrierline_moves() so that only the earliest BT is captured by ADE. This was the result of a bug caused by a carrier clocking out and then clocking in a new BT in the middle of the day.
	Also a bug was reported where the pdfconverter recorded the carrier's name as "BT" instead of the actual name as well as a supervisor's employee id. This was caused by a late occurance of the text "Employee ID" in the raw data created by pdfminer on a particular page. Instead of altering the pdf converter, which works properly 99% of time. I installed a fix in at kbcsv_repair.py>  CsvRepair> testfobadname() which detects if the carrier name is "BT" or "MV" and replaces that which the last valid name seen (which is held in a buffer). The employee id number is also replaced by information in that same buffer. 
	There was also an issue with a an "ATTENTION" notification generated by the SpeedSheet precheck. In cases where there is more than one list status for the week, an inappropriate notificaion might be issued. This is caused by the program only using one list status instead of matching the appropriate list status for the particular date. I didn't fix this, instead I amended the notification to read "Ignore this message if there are multiple list statuses for the week" at kbspeedsheets.py> SpeedRingCheck> check_codes(). 
	The cells holding carrier information (in contrast to ring information) in SpeedSheets was reformated to Text instead of General Numbers. This will allow users to enter a route number such as "0915" without it being changed to "915" by the spreadsheet program. 
	Some return commands were added in kbpdfhandling.py> PdfConver> PageAnalysis to prevent crashing when a Permission or Attribute Error occurs. 

Version 5.06 (01/13/2023)
	Carrier Status History was changed to Carrier History. A new button was added for each carrier in the the results shown in Carrier History. This button labeled "Rings" will generate a report showing all rings for the carrier in the klusterbox database. 
	Updating informal c checks to allow white space before or after the grievance number, and dates. The white space is stipped out before being input into the database, but whitespace before or after those values does not trigger an error.  
	New fields were added to New Carrier and Edit Carrier screens for Employee ID and Seniority Date. The employee id uses the existing name_index table and seniority date uses a new table called seniority. I added classes/ methods in kbtoolbox to check new input for employee ids and seniority dates. I also created two reports. One only displays seniority date by order of seniority, the other includes employee id number and a seniority rank and is listed in alphabetical order. 
	Added moves preference - route first option for speedsheets. This allows the user to opt for the route to appear first in the moves triad when using speedsheets. There is an option in the speedsheet configuration screen which allows the user to turn off/on this option. 
	The Speed Sheet Cheat Sheet is now generated by python code as opposed to being a txt file which is opened by pressing a button. Court Leave was added to the TACS Cheat Sheet. 
	An option was added to the Overmax spreadsheet to allow the user to use 12 hours as a limit to wprk hours as opposed to 11.50 for non-otdl carriers in reference to daily hour violations. The option is selected by going to the Management > Spreadsheet Settings > 12 and 60 Hour Violation Spreadsheet > 12 Hour Simplification Option. 
	I added functionality to OverMax Spreadsheet so that violations for carriers on the work assignment list occur after 12 hours instead of 11.50 hours. There is a setting in Spreadsheet Settings that allows the user to choose between 12 hours or 11.50 hours as the work limit. I added a December Exemption field in to the 12 and 60 Hour Violation Spreadsheet. The user must manually enter 'yes' into the field to alter the formula. This feature voids all violations for otdl carriers when the December Exception is in effect. The user can select the WAL December Exemption setting to void all work assignment carriers during the December Exception. The formulas are generated with variables to reflect the setting specified by the user. 

Version 5.07 (04/16/2023)
	Created a module to allow 2 different klusterbox databases to be merged. This is not assessable through klusterbox, but is placed in the project folder. This allowed me to merge my 2018 db with my 2022 db. 
	In InformalC, I added a button in the search results labeled carrier list which will generate a spreadsheet which list all carriers awarded any remedy in the search criteria and will give their employee id number. 
	Revised the carrier history to reflect the carrier list consistant with the investigation range. Added a restore button to update the carrier so that he is returned to the station. I also added an Insert button for cases where a carrier with no present record, but records in the future can be added to the current investigation range. 
	
Version 5.08 (10/03/2023)
	InformalC is greatly expanded in this release. InformalC now can be updated with speedsheets and the functionality of the graphic user interface offers more. The database has been updated so that more data can be tracked. Added tables in the db are informalc_awards2, informalc_batchindex, informalc_decisioncategories, informalc_gats, informalc_grievances, informalc_issuecategories, informalc_nonindex, informalc_remandindex and informalc_settlements. There is an update in the kbfixes module to move any data in informalc_grv and informalc_awards to new data structures and then drop informalc_grv and informalc_awards. The search criteria screen is much more expanded allowing the user to narrow their search by any criteria and sort by a several dates. Search results are limited to a fixed amount per page, to cut down on time needed to load the screen. A navigation bar allows the user to go to where they want to go in the results. From the search results screen, the user can edit grievance/settlement information, generate a report on a specific grievance or enter monetary awards for the settlement per carrier. The enter awards screen has been updated from earlier releases of informalc to allow the user to track gats descrepancies - cases were management has not paid the settlement in full per the gats report. With this information, informalc can generate reports showing how much is owed to carriers when payment has not been made in full. Several other reports have been created to display information stored in the informalc database tables. 
	
Version 6.00 (12/06/2023)
	The spreadsheet for weekly availability for OTDL carriers was added for this release. It displays the total hours and paid leave for an otdl carrier on a given week and calculates how much availability they have for that week until they get to the 60 hour weekly limit or the 12 hour daily limit. The formulas will zero out availability if paid leave is given for the day. 
	The version of python used was updated to 3.11.6 (from 3.7) so to facilitate upgrading the Pillow package to v.10 (used to display images). The eariler package was flagged as a security risk by github. None of the other requirements were updated. Due to formatting differances, I am retaining python 3.7 for the mac version. I will discontinue the 32 bit version since python 3.11 is not compatable with Windows 7 - which was used to create the 32 bit version. 
	There was also a modification to the MakeWindow() method in kbtoolbox.py which was a fix for the mousewheel binding. 
	I modified the navigation for the MacOS distribution since any swapping of frames in the Tkinter root would quickly result in crashes if those commands originated from a Tkinter menubar (this bug was only present in MacOS). I replaced any such navigation with a Navigation button which would go to a screen showing buttons navigating the app. This makes the MacOS app stable. 
	There was also a fix for the automatic data entry as it relates to collecting data for the level (either 1 or 2). The method find_secondlevel() was modified so as to not close until a value was found. 	
	I created a python module called AliasMaker.py. This is not a dependancy for Klusterbox, but instead it is used to alter the names in the mandates.sql file so that tutorials can be run showing carrier data where the names have been changed. 
	
Version 6.01 (02/06/2024)
	For Automatic Data Entry, I updated kbcsv_repair so that duplicate first lines containing "TAC500R3" or "Employee Everything Report" do not appear in the fixed proxy csv file. Duplicate header lines were causing the Automatic Data Entry to crash. 
	Updated the automatic data entry so that moves to "722" or "744" could be identified as 'return to station' values. 
	Updated CarrierInput class so that blank seniority values delete the record in the database records with no seniority date in the db were causing reports containing seniority dates to fail to open. For informalc reports, I modified the awardstack to show the 'docs?' status along side the awards and gats descrepancies. 
	Created a fix for the this version in kbfixes where all blank senior_date values in the seniority table where deleted. 
	
Version 6.02 (04/04/2024)
	I added remedy functionality for three spreadsheets - the improper mandates spreadsheet, the 12 and 60 hour violations spreadsheet (also called overmax) and the off bid violations spreadsheet. This includes new settings options in under the spreadsheet settings menu which allows the user to toggle off/on the remedy option and also allows the user to enter an hourly remedy rate - a flat dollar amount used to calculate the correct monetary remedy. The hourly remedy rate can also be updated on the spreadsheet itself. 
	I also fixed a bug to display the settlements with gats descrepancies in the only descrepancies report in informalc. 
	
Version 6.03 (06/19/2024)
	I updated the automatic data entry so that will add a record for otdl carriers' nsday. I wrote a method that detects if that ns day is adjacent to a day of annual or sick leave - in those cases the ns day is assigned a code of 'annual' or 'sick'. Otherwise, the ns day is given the code 'no call'.
	The pdf reorder class was made to edit an employee everything report so that the user can specify which carriers are included in a limited and curated employee everything report. The pdf reorder also sorts the report alphabetically by name. 
	The remedy settings and formulas for the improper mandates spreadsheet and the overmax spreadsheet were updated to allow for tolerances. Settings for "impman_remedy_tolerance" and "overmax_remedy_tolerance" were inserted into the tolerances table to hold values for those tolerances.

Version 6.04 (07/28/2024)	
	The ot availability spreadsheet was updated to include cumulative ot and ot availability up to 20 hours. A box was added for the daily hours column to indicate ns days. Once 20 hours of ot is reached, the carrier is no longer available for overtime. 
	
Version 6.05 (07/31/2024)
	The otdl availability spreadsheet formula for daily availability was modified so that ns days when the carrier does not have availability for the 8 hours (the guaranteed time) will show no availability. 
	
Version 6.06 (10/05/2024)
	The Improper Mandates number 5 spreadsheet was added. These spreadsheets were meant to be copy/pasted into contentions for improper mandate grievances. These spreadsheets have the added funtionality of calculating the remedy for otdl and auxiliary carriers as a remedy to be paid at ot rate and remedy to be paid at the penalty rate. 
	The improper mandates remedy spreadsheet was modified so that availability is determined by all factors, including the 20 hour weekly overtime limit, the 60 hour weekly limit, etc
	
Version 6.07 (11/29/2024)
	A fix was made to the pdf converter. pdf miner was reading the level and putting two levels (base and temp) as one line. The formula for the weekly violations on the overmax worksheet was modified so that weekly violations for both aux and ptf carriers are zero in all cases. Page breaks were inserted between the list on the remedy sheet of the improper mandates worksheet. Off bid violations were factored into the calculations for maximum availability on the remedy spreadsheet of the improper mandates worksheet. 
	
Version 6.08 (01/30/2024)
	A setting was added to the Spreadsheet Settings for the offbid violations report which allows the user to show any sundays. "Off" is the default setting, which will result any any clock rings for sunday not being displayed on the off bid violation report. The "on" setting will result in sunday off bid violations being shown as long as the "ns day" code is not in the carrier's clock rings. 
	Added a bug fix for the kbpdfhandling module - LineAnalysis().alt_founddays() function. Wednesday was missing in the tuple of days. 
	I updated the kbspreadsheets module - ImpManSpreadsheet().display_formulas_non() with a new formula for OT to show overtime on the carrier's own route, excluding OT off route. I also updated and expanded the summary sheet to include totals adjusted for availability and tolerances. 
	The Improper Mandates workbook was modified to include a remedies worksheet for 10 hour availability vs ot worked on carriers own assignments - this is a suplement to the remedies worksheet for overtime worked off of a carrier's route. Exposition was added to the front of the remedies sheets to define the remedies sheets. Exposition was also added to the reference sheet to define the tolerances. The order of the sheets was changed so that the summary and reference sheets are at the end of the workbook. 
	Upgrades were made to the formating of the 12 and 60 Hour Violations Worksheet. 
	
Version 6.09 (03/16/2025)
	The pdf converter was modified to allow for variable input for the jobs array. The csv repair module of the automatic data entry which generates the proxy file was modified so that errors in the csv file result in the problem carrier information being aborted - allowing the full report, minus the one carrier, to be generated. 
	
Version 7.00 (07/02/2025)
	Changes were made to comply with the 2023-2026 National Agreement which takes effect on July 1, 2025. New list statuses were added: "odlr" (overtime desired list regular day only) and "odln" (overtime desired list non scheduled day). Input modalities: edit carrier information dialog, multiple input, speedsheets and automatic data entry were updated. All related checks were updated. The improper mandates spreadsheets were updated. Some formatting on the overmax spreadsheet was updated. the otdl weekly availability worksheet was modified so that both odlr and odln appear on the worksheet. The otdl equitability spreadsheet was modified to track either otdl, odlr or odln overtime over the quarter. The otdl preference db table was altered to add make up opporunities for odlr and odln list. A fix was written into the kbfixes module to make all related null values into empty strings. A method was added to eliminate the menu option for the 10 hour preference if the investigation range is after July 1st. 
	
	Wrap text was added to some column headings in the improper mandates number 4 spreadsheet. 
	
	Fixed a bug with the more moves button in the kbenterrings module. The widget configuration was adjusted so that the appropriate widgets appear on days when odln and odlr are considered available for overtime or not available for overtime. 
	
Version 7.01 (09/01/2025)
	CsvRepair().test_write() was modified so that the final cache of information is processed. Before the class was not including the last carrier in the results. 
	
	PdfConveter().WriteCsv().fix_jobs() was modified to rephrase joint.remove(-1) to del joint(-1). The previous as not removing the last element of the array and causing a crash of the pdf converter. 
	
	Automatic data entry was modified so that the default list status for carriers is 'nl'. The index of the value changed with the adding of other list statuses to the array of list statuses. 
	
	In Automatic data entry, the staticmethod _get_nsdays() was modified so that an arguement of none would return an empty string immediately. This was causing a crash since 'none' was not included in the projvar.nsday array. 
	
	The OtEquitability was modified with the self._overtime_max() method so that for odlr equitability maxes out at 4 hours and odln overtime maxes out at 8 hours. "otdl" equitability was left unmodified. 
	
Version 7.02(09/27/2025)
	The OtEquitability was modified with the self._overtime_weekly_max() method so that odlr and odln equitability stops incrementing after the carrier has worked 60 hours in a service week. 
	
	An informal c report was added which generated a list of grievance numbers only. 
	
	The automatic data entry defaults were adjusted due to adding of new list statuses. InformalCReports().grv_summary() was modified to take a kwarg allowing for a short report. 
	