I feel the same way about not wanting to give my credentials out. I built a tool that takes all the Bank Expenses CSV exports and aggregates then I to a single CSV.
I don't know if that's what you're looking for, but it may help you out.
I would be extremely surprised if there was one, as there are about 70000 banks in the US.
Some of the large banks now have APIs to access their data, but those aren't publicly available.
Most smaller banks and credit unions don't have a public API, so apps like Mint rely on web scraping. That's a huge and expensive undertaking, considering that web sites change constantly and all the scraping could has to be adjusted for it.
Companies like Intuit are willing to invest the time it takes to keep up because they can sell your data (I guess) or sell you additional products. I don't see how the open source community could keep up with this.