I have a 700 mb XML file that I process from a records tree to an EDN file.
After having do all the processing, I finally have a lazy sequence of hashmaps that are not particularely big (at most 10 values).
To finish, I want to write it to a file with
(defn write-catalog [catalog-edn] (with-open [wrtr (io/writer "catalog-fr.edn")] (doseq [x catalog-edn] (.write wrtr (prn-str x)))))
I do not understand the problem because
doseq is supposed to do not retain the head of the sequence in memory.
My final output
catalog is of type
I then do
Then memory usage is grinding and I have a GC overhead error at around 80mb of file writter with a XmX of 3g.
I tried also with a
spit and no
prn-str, same thing happen.
Is this a normal behaviour ?