Lecture 4: Knowledge Representation | Môn Data Mining - Trường Đại học Quốc tế, Đại học Quốc gia Thành phố Hồ Chí Minh

lOMoAR cPSD| 23136115 2/22/2025 Lecture 4: Data mining knowledge representation
Lecturer: Dr. Nguyen, Thi Thanh Sang (nttsang@hcmiu.edu.vn) References:
Chapter 3 in Data Mining: Practical Machine Learning Tools and Techniques
(Third Edition), by Ian H.Witten, Eibe Frank and Eibe Frank 1 2 lOMoAR cPSD| 23136115 2/22/2025 Knowledge representation Tables Linear models Trees Rules Classification rules Association rules Rules with exceptions More expressive rules
Instance-based representation Clusters 3 3 lOMoAR cPSD| 23136115 2/22/2025 Outlook Humidity Play Sunny High No Tables Sunny Normal Yes Overcast High Yes ► Simplest way of Overcast Normal Yes representing output: Rainy High No ► Use the same format as input! Rainy Normal No
► Decision table for the weather problem:
► Main problem: selecting the right attributes 5 5 Linear models
Another simple representation Regression model lOMoAR cPSD| 23136115 2/22/2025
Inputs (attribute values) and output are all numeric
Output is the sum of weighted attribute values
The trick is to find good values for the weights 7 7
Linear models for classification Binary classification
Line separates the two classes lOMoAR cPSD| 23136115 2/22/2025
Decision boundary - defines where the decision changes from one class value to the other
Prediction is made by plugging in observed values of the
attributes into the expression
Predict one class if output 0, and the other class if output < 0
Boundary becomes a high-dimensional plane
(hyperplane) when there are multiple attributes 9 9 Knowledge representation Tables Linear models Trees lOMoAR cPSD| 23136115 2/22/2025 Rules Classification rules Association rules Rules with exceptions More expressive rules
Instance-based representation Clusters 11 11
Nominal and numeric attributes ●Nominal:
number of children usually equal to number values
attribute won’t get tested more than once
●Other possibility: division into two subsets lOMoAR cPSD| 23136115 2/22/2025 ●Numeric:
test whether value is greater or less than constant attribute may get tested several times
●Other possibility: three-way split (or multi-way split)
●Integer: less than, equal to, greater than ●Real: below, within, above 13 13
Does absence of value have some significance?
“missing” is a separate value
Solution B: split instance into pieces Trees for numeric prediction
Regression: the process of computing an expression that predicts a numeric quantity
Regression tree: “decision tree” where each leaf predicts a numeric quantity
Predicted value is average value of training instances that reach the leaf lOMoAR cPSD| 23136115 2/22/2025
Model tree: “regression tree” with linear regression models at the leaf nodes
Linear patches approximate continuous function 15 15 lOMoAR cPSD| 23136115 2/22/2025
Regression tree for the CPU data 17 17 lOMoAR cPSD| 23136115 2/22/2025 Knowledge representation Tables Linear models Trees Rules Classification rules Association rules Rules with exceptions More expressive rules
Instance-based representation Clusters 19 19 Antecedent
Consequent conclusion): classes, set of classes, or
probability distribution assigned by rule lOMoAR cPSD| 23136115 2/22/2025 From trees to rules
●Easy: converting a tree into a set of rules One rule for each leaf:
●Antecedent contains a condition for every node on the
path from the root to the leaf
●Consequent is class assigned by the leaf
●Produces rules that are unambiguous
Doesn’t matter in which order they are executed 21 21 lOMoAR cPSD| 23136115 2/22/2025
A tree for a simple disjunction 23 23 lOMoAR cPSD| 23136115 2/22/2025
A tree with a replicated subtree If x = 1 and y = 1 then class = a If z = 1 and w = 1 then class = a Otherwise class = b 25 25 lOMoAR cPSD| 23136115 2/22/2025 Interpreting rules
●What if two or more rules conflict? Give no conclusion at all?
Go with rule that is most popular on training data? …
●What if no rule applies to a test instance? Give no conclusion at all?
Go with class that is most frequent in training data? … 27 27
Assumption: if instance does not belong to class “yes”, it
Order of rules is not important. No conflicts! lOMoAR cPSD| 23136115 2/22/2025 Knowledge representation Tables Linear models Trees Rules Classification rules lOMoAR cPSD| 23136115 2/22/2025 Association rules Rules with exceptions More expressive rules
Instance-based representation Clusters 29 29 lOMoAR cPSD| 23136115 2/22/2025 Knowledge representation
Support and confidence of a rule
Support: number of instances predicted correctly
Confidence: number of correct predictions, as proportion of all
instances that rule applies to
Example: 4 cool days with normal humidity
If temperature = cool then humidity = normal lOMoAR cPSD| 23136115 2/22/2025
Support = 4, confidence = 100%
Normally: minimum support and confidence prespecified
(e.g. 58 rules with support 2 and confidence 95% for weather data) 31 31 lOMoAR cPSD| 23136115 2/22/2025 Knowledge representation Tables Linear models Trees Rules Classification rules lOMoAR cPSD| 23136115 2/22/2025 Association rules Rules with exceptions More expressive rules
Instance-based representation Clusters 33 33 33