So it sounds like you were affected still before the start of Panda #22, but seems like you were affected on the later edge of that ghost update. You really should participate in this thread since you also run a UGC website being hurt:
https://productforums.google.com/forum/#!msg/webmasters/T-4i0yB7CrQ/M7mngz551hgJ
JonhMu has already responded, and the thread seems to be gaining a bit of attention. I think with you posting there it might grab some more positive attention since you were a poster child for being affected once before. I really think your site, my site, and some of these other UCG type websites are in a different group that were affected. I don't feel we fit in with the regular types of sites that were affected by Penguin and Panda.
It would at least be nice to get an answer to what is going on if anything.