How MySQL Used in Big Data Analysis

MySQL is efficient around handling highly concurrent accesses to transactional data on a single machine. MySQL has no way for actual work with big data storage and retrieval. At some point, we realize that there are some matured database systems. For example, it is not abnormal to face scaling issues in Python. SQlite comes with Python. If data is large then there are some NoSQL database such as MongoDB. Yet, we see MySQL to be used in Big Data works. How MySQL Used in Big Data Analysis?

How MySQL Used in Big Data Analysis?

MySQL can be used with traditional big data system like Hadoop. Large amounts of data can be stored on HDFS and also processed with Hadoop. Raw metrics might be stored in HDFS. While the output can be stored on the MySQL server for analysis. The aggregated data can be saved in MySQL. MySQL itself can be used as a big data store. There are sharding software which supports parallel queries over shards. Shard-Query works better with InnoDB, TokuDB. As for MySQL, InnoDB’s index-organized structure has useful advantages for big data analysis which are more useful for analytics. So, for analytics, we can build structures optimized for specific analysis. Alternatively, Infobright, InfiniDB like storage engines can be used. When we are using MySQL in the ways stated, that is probably not data which is actually big data. We are using MySQL simply because that is easy, mostly easier to integrate with many traditional tools, scripts for graphing.

MySQL itself has small article :

Percona had webinar on implementing MySQL and Hadoop for Big Data :

As possibly a sane conclusion, we can say that for Big Data, MySQL is still a valid solution. But where the other issues arises or need is obvious such as high availability, extreme fault-tolerance, easy scalability, concurrency and reliable automatic replication among databases, it is probably better to choose some other tool.