A Case Study for NoSQL Applications and Performance Benefits: CouchDB vs. Postgres
As HPC continues driving towards exascale, there is an in- creasing need to access and interface with massive data sets. As such interactions become more commonplace, developers need to readdress how data is stored and rethink the in- frastructure to enable efficient access and analysis of data. It is becoming evident that traditional, relational databases are not always the most appropriate solution to allow users on-demand access to big data sets. In this study we show that using non-relational, ”NoSQL” databases, such as doc- ument stores or key-value stores, can offer large benefits in performance, accessibility and availability. We present a use case from the TeraGrid User Portal that demonstrates a NoSQL solution for auditing and processing batch job data efficiently in order to provide users rapid access to this data. We also present an analysis of benchmarking data to iden- tify usage patterns and data sets for which each architecture is best suited.