Apache Hadoop is a framework for big data. One of its main components is HDFS, Hadoop Distributed File System, which stores that data. You might expect that a storage framework that holds large quantities of data requires state-of-the-art infrastructure for a file system that does not fail, but quite the contrary is true.

Developing a tool that runs on artificial intelligence is mostly about training a machine with data. But you can’t just feed it information and expect AI to wave a magic wand and produce results. The type of data sets you use and how you use them to train the tool are important. Here are five pitfalls to be wary of.

Apache Sqoop, Flume, and Kafka are tools used in data science. All three are open source, distributed platforms designed to move data and operate on unstructured data. Each also supports big data in the scale of petabytes and exabytes, and all are written in Java. But there are some differences between these platforms.