javascript - Concatenating millions of xml files in a single json file using bash -
i merge 2.5 million smallish xml files directory tree 1 large json file, , trying using bash using find
, xml2json
utility.
i'm pretty new bash , haven't done complicated it. intuition following (but long way working):
find . -exec xml2json {} ; cat >> merged.json
problem #1: can't figure out how use xml2json utility -exec
.
find . -exec /usr/bin/xml2json < {}
doesn't work (seems it's waiting more input?). neither
find . -exec /usr/bin/xml2json {}
how working?
problem #2: efficient way concatenate files? using cat
isn't going create well-formed json file, can concatenate in brackets @ start , end , commas in between? or should use jq
's -s
? need stream or parallelize it?
if turns out bash bad this, efficient alternatives in javascript, r, or python useful. thanks.
Comments
Post a Comment