Looking into solving a very large data set problem by using bit manipulation to reduce compute Imagine billions of records that look like this(but more than 20k in length): Questions : A,B,C,D Answers : 0 1 0 1 The length of questions and answers is always the same, the answers is stored in true/false as bits if the answer was correct I get inputs like this (A & B) and I need to find the records that have A and B as true. it can get into the weeds with for ex (( A & B ) ^ C ) | D My idea in theory would be to store the questions as the bits that would match up with the answer equivalent A 1000 B 0100 C 0010 D 0001 so possibly A & B would be 1100 as required bits to be flipped for an input to be considered true but this also includes these cases as being true 1100 1110 1101 1111 where I start doubting this is when I get into the ors and nots. Cant wrap my head around how to evaluate those. I am also open to other suggestions. Right now the best idea I have that is more sound would be processing the questions as key:question val:index 1 time and then turn the answers into a similar state and evaluate from there Continue reading...
