gzip file with splitting the record into columns when the one of the column value in double quote

By : Bhaskar
Source: Stackoverflow.com
Question!

I have gzip file which contains columns separated by comma, but when the column value is within double quotes the commas should be kept as it is. I wrote the following code:

                           input = gzip.open(file, "rb")
                            reader = codecs.getreader("utf-8")
                            input_file = reader(input)
                            try:
                                count = 0
                                for line in input_file:

                                    try:
                                        # print 'count='
                                        # print count
                                        if len(line) != 0:

                                            col = line.split(',')

My data in the file looks like:

4798151,1137351,nam_p0,2762913,nam_r000,"NAM_Rack, Power & Cooling",3
4798151,1135623,nam_s0,2762914,nam_a0,"NAM_Advise, Transform & Manage",3

When I was splitting data with comman, the comma with in double quotes should ignore and come into a column. I am not sure how to add the condition treating the text enclosed in double quote as one. A quick response would be a great help. Thanks.

By : Bhaskar


Answers

Use csv.

Demo

>>> import StringIO
>>> import csv
>>> line = '4798151,1137351,nam_p0,2762913,nam_r000,"NAM_Rack, Power & Cooling",3'
>>> handler = StringIO.StringIO(line)
>>> [row for row in csv.reader(handler, delimiter=',')]
[['4798151', '1137351', 'nam_p0', '2762913', 'nam_r000', 'NAM_Rack, Power & Cooling', '3']]

In this case you can use this direct approach:

with gzip.open(file, 'rb') as handler:
    for row in csv.reader(handler, delimiter=","):
        # row processing HERE
By : klashxx


You have to get the collection object first as mentioned below

  var collection = db.collection("simple_document_insert_collection_no_safe");

// Insert a single document collection.insert({hello:'world_no_safe'});

Please refer the below link for more details.

mongod db collection using nodejs



I think you could rewrite this as a single query on the status table, using ORs to test each scenario, like so:

SELECT ss.int_tran_id
FROM   status SS
WHERE  ss.stage in ('ACHPayment_Confirmed', 'HIFV4', 'HIFV5_FTRINF', 'Payment_HIFV5_FTRINF')
OR     (ss.stage = 'PREVDAY'
        AND (EXISTS (SELECT NULL
                     FROM   references rf
                     WHERE  ss.int_tran_id = rf.int_tran_id
                     AND    rf.mid_ref IS NOT NULL)
             OR EXISTS (SELECT NULL
                        FROM   app_data ad
                               INNER JOIN ach aa
                                 ON (ad.app_data_id = aa.ach_id)
                        WHERE  ss.int_tran_id = ad.int_tran_id
                        AND    aa.par_number IS NOT NULL)));
By : Boneist


This video can help you solving your question :)
By: admin