COMP 3002 Applications of Big Data

This is an archived copy of the 2022-2023 catalog. To access the most recent version of the catalog, please visit https://hbook.westernsydney.edu.au.

Credit Points 10

Legacy Code 301110

Coordinator Yi Guo Opens in new window

Description Many techniques and tools have been developed over the past decade to cope with the ever-growing needs for the processing and analysis of big data. This subject will cover the key techniques that have been widely used in big data applications, such as relational and Not Only Structured Query Language (NoSQL) databases, Web Services, parallel and cloud computing, MapReduce, Hadoop and its eco-system. It aims to introduce the emerging technologies and applications in big data to students, and keep up with the latest trends in the industry.

School Computer, Data & Math Sciences

Discipline Computer Science

Student Contribution Band HECS Band 2 10cp

Check your HECS Band contribution amount via the Fees page.

Level Undergraduate Level 3 subject

Pre-requisite(s) COMP 1013 OR COMP 1005

Assumed Knowledge

Knowledge of computer software, databases, and entry-level statistics.

Learning Outcomes

On successful completion of this subject, students should be able to:

Explain the major trends and latest development in big data technology
Describe a selection of major techniques in use today for big data storage and processing, including NoSQL, MapReduce, cloud computing, web services
Build tools to obtain data from various sources and in different formats
Evaluate the relative strengths and limitations of relational and NoSQL database systems and recommend the most appropriate solution to data storage and access for different application scenarios
Employ HDFS and Hadoop for data storage and manipulation on parallel platforms
Select and apply appropriate tools from the Hadoop eco-system for big data management tasks

Subject Content

Sources and formats of big data
Relational Databases and SQL
NoSQL Databases
Web Scraping
Cloud computing platforms for big data
Data parallelism and the MapReduce framework
Data storage and processing with Hadoop Distributed File Systems (HDFS) and Hadoop
The Hadoop eco-system, including Pig, Hive, Spark, etc.

Assessment

The following table summarises the standard assessment tasks for this subject. Please note this is a guide only. Assessment tasks are regularly updated, where there is a difference your Learning Guide takes precedence.