Unit 1: AI Project Cycle & Ethics
1. The 5 Framework Stages
- Problem Scoping: Identifying a real-world problem and defining goals using the 4Ws Canvas:
- Who? (Stakeholders affected)
- What? (Nature of the problem & evidence)
- Where? (Context/location of the problem)
- Why? (Benefits of solving it)
- Data Acquisition: Collecting relevant datasets from reliable sources (surveys, web scraping, sensors, APIs). Key factors: Data quality, accuracy, and volume.
- Data Exploration: Cleaning, organizing, and visualizing data using graphs/charts to discover patterns and outliers before building models.
- Modeling: Selecting and training machine learning algorithms on the collected data.
- Evaluation: Testing the model on unseen data to assess reliability, precision, recall, and accuracy metrics.
2. AI Ethics & Concerns
- Data Privacy: Ensuring personal user data collected by algorithms is stored securely and ethically obtained.
- AI Bias: Algorithmic bias occurs when training data contains existing human prejudices, leading to unfair decisions.
- AI Access: The digital divide between those who have access to advanced technology and those who do not.
Unit 2: Advanced Modeling
1. Rule-Based vs. Learning-Based Approaches
- Rule-Based Approach: The developer manually codes explicit rules and instructions into the system. The machine does not learn independently.
- Learning-Based Approach: The machine is fed data and answers, allowing it to discover underlying rules and patterns on its own.
2. Core Machine Learning Types
- Supervised Learning: Trained on labeled data (Input + Target Answer). Examples: Classification (spam vs. non-spam) and Regression (predicting home prices).
- Unsupervised Learning: Works with unlabeled data to find hidden structures. Examples: Clustering (customer segmentation) and Dimensionality Reduction.
- Reinforcement Learning: Agent learns by trial and error using a system of rewards and penalties.
3. Neural Networks & Perceptrons
Artificial Neural Networks (ANNs) mirror human brain structure. A Perceptron is the basic processing unit (single-layer neuron):
- Takes inputs ($x_1, x_2$), multiplies them by weights ($w_1, w_2$).
- Adds a bias value ($b$).
- Passes the sum ($Z = \sum x_i w_i + b$) through a step activation function to output 0 or 1.
Unit 6: Natural Language Processing (NLP)
1. Text Normalization Pipeline
- Sentence Segmentation: Breaking a large text block into individual sentences.
- Tokenization: Splitting sentences into discrete units called tokens (words, numbers, or symbols).
- Removing Stopwords: Filtering out high-frequency words that carry little semantic value (e.g., "is", "the", "and").
- Converting Case: Normalizing all text to lowercase to prevent duplicates ("AI" vs "ai").
- Stemming / Lemmatization: Reducing words to root forms:
- Stemming: Chopping off affixes algorithmically (e.g., "crying" -> "cry"). Fast but can produce non-words.
- Lemmatization: Uses a dictionary to reduce words to meaningful root words (e.g., "crying" -> "cry", "better" -> "good").
2. Bag of Words (BoW) & Document Vector Table
BoW converts unstructured text into numerical feature vectors by counting word frequency across a corpus, ignoring word order and grammar.