Definition
The Data Science Life Cycle (DSLC) has six steps: (1) understanding the problem, (2) collecting data, (3) cleaning data, (4) analyzing data, (5) visualizing data, (6) interpreting data.
The first step in the DSLC is understanding the problem — clearly defining exactly what question needs answering before doing anything else.
Definition
Understanding a problem before collecting data is important because it determines exactly what data actually needs collecting, and how — without a clear problem definition, you risk gathering data that's irrelevant, insufficient, or in the wrong form to actually answer the question at hand, wasting effort and potentially leading to wrong conclusions.
'Interpreting Data' in the DSLC means drawing meaningful conclusions and practical insights from the analyzed and visualized data — connecting the results back to the original problem to answer it and, often, to recommend an action.
Definition
Two methods of collecting data: surveys/questionnaires (e.g. asking students about their study habits) and sensors/automated devices (e.g. a smart water meter automatically recording water usage).
Surveys are best for collecting opinions from many students, since they can be distributed widely and quickly, letting many people's views be gathered efficiently.
Data collection using devices like smart water meters or wearables (which automatically record data like heart rate or activity level) is a sensor-based/automated collection method.
Automatically collecting public data from websites using a program is called web scraping.
Definition
An example of a data source: a school's student record database, a government census dataset, or sensor readings from IoT devices.
When choosing a reliable data source, you should check its accuracy, how up to date it is, and whether it comes from a credible, trustworthy origin — an unreliable source can introduce errors or bias into every later stage of the analysis.
Definition
Bias in data is a systematic skew or distortion that makes collected data unrepresentative of the true situation it's meant to reflect — e.g. surveying only one age group when studying an opinion held across all ages. Bias affects results by making conclusions drawn from the data misleading or inaccurate, since the data doesn't genuinely reflect the full picture it's assumed to represent.
Definition
'Cleaning' data means finding and fixing (or removing) errors, inconsistencies, and irrelevant entries in a dataset, so that later analysis is based on accurate, usable data.
The difference between 'missing data' and 'outliers': missing data is where a value simply wasn't recorded at all (a gap in the dataset), while an outlier is a value that was recorded but is unusually different from the rest of the data (either a genuine unusual case or a data-entry error) — both are common data errors, but one is an absence, the other an oddity.
It's important to validate data before using it because unvalidated data may contain errors that, if unnoticed, would silently produce wrong analysis results and unreliable conclusions — validation catches these problems before they affect the rest of the process.
Example 1 — cleaning a messy survey
Give two steps to follow when cleaning a messy survey.
- Check for and remove or correct duplicate entries (e.g. the same person's response submitted twice).
- Check for and handle missing or inconsistent answers (e.g. deciding whether to discard incomplete responses or fill in reasonable estimated values).
Answer: (1) Remove/fix duplicates, (2) handle missing or inconsistent responses
Definition
The main goal of analyzing data is to find meaningful patterns, trends, and relationships within it, that help answer the original problem/question.
Definition
The purpose of visualizing data using charts is to present patterns and relationships in a visual, quickly understandable form — a well-chosen chart can reveal a trend or comparison far more immediately than scanning raw numbers in a table.
Comparing chart types: a bar chart is best for comparing distinct groups or categories against each other (e.g. sales figures for different products); a pie chart is best for showing how a whole is divided into proportional parts (e.g. percentage market share); a line chart is best for showing how a value changes continuously over time (e.g. temperature over a week).
Definition
Two tools used for creating charts: spreadsheet software like Microsoft Excel or Google Sheets (ideal for beginners, simple and widely available), and dedicated data visualization platforms like Tableau or Power BI (more advanced, offering richer, more customizable visualizations for larger or more complex datasets).
Remember
Two reasons data visualization is important: it makes patterns and trends far easier and faster to spot than raw numbers alone, and it communicates findings clearly to an audience (including non-technical people) who might not otherwise easily interpret a table of raw data.
Definition
Before storing data, it should be cleaned and validated, to ensure only accurate, usable data ends up in permanent storage.
A relational database stores information in the form of tables (rows and columns), with related tables linked together through shared key fields.
In a database, a single row of data (e.g. one student's complete record) is called a record. A 'field' in a database table is a single specific category/column of data within each record — e.g. in a student table, 'Name', 'Roll Number', and 'Class' would each be separate fields.
DBMS (Database Management System) is the software used to create, manage, and interact with a database as a whole (e.g. handling storage, security, and access). SQL (Structured Query Language) is the specific language used within a DBMS to write commands/queries for retrieving, inserting, updating, or deleting data — a DBMS is the overall system, while SQL is the tool/language used to communicate with it.
Example 2 — writing a simple SQL query
Write an SQL query to get all data of a student whose roll_number is from the `student` table.
- `SELECT` chooses which columns to retrieve — `*` means all columns.
- `FROM student` specifies the table to search.
- `WHERE roll_number = 15` filters for only the matching row.
Answer: SELECT * FROM student WHERE roll_number = 15;
Definition
The SQL command used to add a new entry to a table is `INSERT` (specifically `INSERT INTO`).
Definition
Three real-life applications of Data Science: healthcare (analyzing patient data to improve diagnosis and treatment), e-commerce (analyzing customer behaviour to recommend products), and finance (analyzing transaction data to detect fraud).
Quick Recap
- ●DSLC (6 steps): understand problem collect clean analyze visualize interpret data
- ●Collection methods: surveys (opinions, wide reach), sensors/devices (automated, e.g. smart meters, wearables), web scraping (automated public data collection)
- ●Bias: unrepresentative data misleading conclusions; reliable sources need accuracy, recency, credibility
- ●Cleaning: fix/remove errors, duplicates; missing data (absent value) vs outliers (unusual recorded value); validate before use
- ●Analysis finds patterns; visualization: bar (compare groups), pie (parts of a whole), line (change over time); tools: Excel/Sheets (beginner), Tableau/Power BI (advanced)
- ●Database: table rows (records) columns (fields); DBMS (manages the database) vs SQL (query language, e.g. `SELECT`, `INSERT`)
- ●Applications: healthcare, e-commerce, finance (among others)
