The problem: given a 20gb xml file with the following structure <root> <outer> <inner prop="x">...</inner> <inner prop="y">...</inner> </outer> <outer> <inner prop="z">...</inner> <inner prop="f">...</inner> </outer> .... .... </root> How could one xpath count(//outer[inner/@prop="x" and inner/@prop="y"]) efficiently? I have tried xmllint, pcregreg, xmlstarlet, xml_grep in Linux, even awk and grep but I keep getting the system out of memory. I was considering python sax module, but haven't found anything relevant and I also don't know how such an xpath like count could work with streaming. It would also be great if sax could somehow ignore inner text, as the file in questions contains several unescaped characters which render the xml not well formed... Tough one Continue reading...
