facebook twitter youtube
in How & Why - 17 Dec, 2011
by Amit Gupta - no comments

I am working on a project where I need to read a text file and for every valid record I have to update database. I also have to write a response/log file. This seems very easy. Although I decided to make a generic solution which can be followed in object oriented way in any project easily.

Requirement

1.       Read all files from a folder following some specific naming conventions.

2.       There will max N records in a file and M fix number of columns in a record.

3.       File header will contain specific fields which are required to validate records in a file.

4.       There will be 2 tables in database. One for keeping file processing information like number of successfully proceed records, total records in a file, failed record, start & end time etc. And another for keeping records detail.

5.       Each field in record has to be format validated and then business validated.

6.       Instead of any update to database first the file then the record must be validated.

7.       It should generate a response file which can be sent for monitoring purpose later.

Design:

I made some classes where each class is responsible for their work only.

FileProcessor (IO processing)

1.       Reads Input folder for specific type of file.

2.       Calls ValidateFile() of FileValidator

3.       Moves a file to specific folder as per return value from ValidateFile()

FileValidator (Single file validation)

1.       Reads each line from input file.

2.       Calls

  1. isValidate() of FileHeader for first line
  2. isValidate() of FileFooter for last line
  3. isValidate() of FileRecord for rest lines

3.       It also writes a response/log file. You can make a separate class for response file in case you want formatted output/logs.

4.       It also performs database operation, if required in your project, when isValidate() returns specific message.

Status

1.       This is an enum.

2.       FileOperation class takes multiple decision on the basis of return value from ValidateFile(). Same with FileValidator class. It writes different messages to response file as per return value from isValidate(). This decision can’t be taken on the basis of boolean value. I guess returning enum is better than returning a String.

FileHeader, FileRecord, FileFooter (Single record validation)

1.       Header of a file generally contains meta data of file records like number of records in a file, max & min length of record, number of fields in a record line etc.

2.       All of them are made as bean class. It helps to store values in database and to compare later.

3.       Any change to these classes will not impact other classes but enum Status. To remove this dependency or to make it more independent you can create a separate class or interface. But remember FileProcessor and FileValidator classes take decision on the basis of return enum. So if this value get changed a little bit change in these classes will also be required.

Java data File Processing analysis

*This article is copyrighted with all examples and images used in this article. Publication of this article to any where without referencing to original article or to hard copy or any modification in original images or example is strictly prohibited.

Amit Gupta

Hey! this is Amit Gupta (amty). By profession, I am a Software Eng. And teaching is my passion. Sometimes I am a teacher, as you can see many technical tutorials on my site, sometimes I am a poet, And sometime just a friend of friends...

Leave a Reply

captcha