Adaptivepath: Things to do at the beginning..
The Unplugged
Test file of 200Mb(800,000 records)
laptopsandparts4less.com
 
File Merge and Xreference
CSV Record merge in Perl FOR www.laptopsandparts4less.com by JOHN JAN POPOVIC


File Merge and Xreference
Description >

- The file must be processed in less than 30 minutes
INPUT FILES >
1. input file: memory.csv ( unformatted raw data, there are 20 files like this one)
2. formatted data file: hp-parts.csv (record has 6 fields)
OUTPUT FILE >
3. output.csv
------------------------------------------------------------------------------------------------
- for each (Model, PNnumber) in the file: hp-parts.csv, we must search through all (1st to last) memory.csv records
- when we find a match, then we must copy the FOUND_record to an OUTPUT file file and prefix it with the (Product_type,Manufacturer,Family,Series,Model,PN_number)
------------------------------------------------------------------------------------------------
this process must be repeated for all records in the hp-parts.csv file

memory.csv file is 200mb (800,000 records)
there are 20 files similar to memory.csv file

Test file of 200Mb (800,000 records)


IT CAN BE DONE EASILY IN PERL