Complete CDR to MySQL parser Perl script

We need a parser that is run from crontab and parses the newest files from the source directory. About 80% of the project is completed. The parser works, but has no error handling and the file name is hardcoded. The attached script has sensitive data removed, but is otherwise functional. Old project tasks to complete: 1. Read the specification file into a MySQL table. (one time, not automated task) (will be sent after proj accepted) 2. Script writes the file name of the parsed file into a third table. If any errors occurred during the parse, then it writes a binary flag and dumps errors into a BLOB/TEXT. Add one field as a flag that is false if no errors and true if errors. The error messages should be human readable. Required error cases are: 1. file incomplete (all rows are fixed length), 2 row/field contains data that is the wrong type and 3. directory/file inaccessible/unreadable etc. Currently only #3 is implemented in the attached script. If possible, #2 should be implemented using warnings from mysql. e.g trying to cast hex/alphanumeric into integer. The script uses the DBI database interface. This is the most reliable choice for our platform, but unfortunately its warning and error handing sucks. We are not interested in swapping it out for something else. If it is impossible to capture warnings from the DBI writes, then our fallback plan is to read how many rows have been written and the sum of one of the accounting fields. These 2 numbers are compared to the result of the read file. If they match, everything is OK. New tasks: 3. We receive 2 files of identical format. The script should do a loop and read both files. The data in the files are written to different tables, but errors are written to the same log table. 4. Add command line argument that display the proposed tables scheme and another that displays debug/verbose and usage. Normally the script is run from crontab, so it should only print to STDOUT or STDERR if there is an actual error. Crontab logging is complementary to the error logging into the mysql log table (which is my primary way of checking for errors). Except for DB connect errors which can not be logged to mysql. 5. Add deterministic ID for the log and data file because our master-master setup prevents the use of auto-incremented indexes. Lets use a massive index with contingency for up to 99999 row per day. Row 55 of [login to view URL] will be 15021200055. For the "log" table, use 1502121 for cdr_xxx and 1502122 for in_cdr_xxx. Lets use the date from the first row in the file, we assume that there will never be 2 days of data in any file. 6. Add creation of the log table to sub create_table. Bid range = 50 - 100 EUR Bid deadline is Thursday 05/03/2012 midnight EET