Data and Workflow Management for Bioinformatics | BINF.X410

This course explains where large data sets come from and how they are stored and managed. It also examines data sizes, accessibility approaches, and how data are transformed and used for AI consumption. You will examine the challenges and considerations when choosing data for training sets. 

By the end of course, you will understand the types of data used in bioinformatics, how the data are collected, stored, managed and searched, and how the data are transformed for further processing and analysis. You will also develop skills on how to aggregate and normalize the data to be used for machine learning and/or AI training sets. 

Learning Outcomes

At the conclusion of the course, you should be able to

• Identify the different types of data used in bioinformatics, their sources, and how they are collected, stored, searched, and managed.
• Explain how bioinformatics data are processed, transformed, and prepared for further analysis, including machine learning and AI applications.
• Demonstrate skills to aggregate, clean, and normalize bioinformatics data to ensure quality and consistency for AI training sets.
• Analyze the sizes, formats, and accessibility of bioinformatics datasets and understand key storage and management considerations.
• Evaluate the challenges and key considerations in selecting bioinformatics data for AI model training, including data quality, bias, and ethical implications.

Topics Include

• Pipeline Design
• Workflow management systems and workflow analysis with open-source tools
• Documentation skills / proof of concept with foresight
• Using SQL for bioinformatics data
• Data lakes (e.g, Databricks, Redshift and/or Snowflake)
• Large data sets
• Databases - how to store, move, and learn what AI models to use