1) Independence is often defined as p(x,y) = p(x)p(y). Using the
definition of conditional probability p(x|y) = p(x,y)/p(y), show that
the alternate definition p(x|y) = p(x) must be true if p(x,y) = p(x)p(y).
For the following problem please don't round your answers until the
final step, and SHOW ALL STEPS IN YOUR CALCULATIONS!)
2) Congratulations! You've been accepted into Hogwarts' School of
Witchcraft and Wizardry. As you step into the Great Hall, you see a
row of students waiting for the Sorting hat to tell them what House
they'll be living in for the rest of their time at Hogwarts. There
are four houses: Gryffindor, Hufflepuff, Ravenclaw, and Slytherin.
You're told by a passing professor that everyone gets Sorted, and
that the probabilities of being sorted into each house are:
p(G) = .20
p(H) = .30
p(R) = .35
p(S) = .15
As a well-educated Penn student and voracious reader,
you've noticed from perusing a certain set of best-selling chronicles
(about a famous wizard with dorky glasses and a notable scar) that
Main Characters in these stories have a different set of probabilities
of being Sorted in a given house.
In particular, from memory you estimate that:
p(M | G) = 0.4
p(M | H) = 0.3
p(M | R) = 0.1
p(M | S) = 0.2
where p(M | G) means probability of being a Main character given that you're in Gryffindor.
Because you're not only a well-educated Penn student, but one who also
took CIS140, you can figure out the likelihood that you are, in fact,
a main character in this story.
a) Based on the above priors, what is the probability that you are a main
character? That is, what is p(M), before any sorting has happened?
b) That was a silly question; of course you're a main character! Now,
given that fact, what house are you most likely to be sorted into?
That is, calculate p(G | M), p(H | M), p(R | M), p(S | M), and say
which is most likely. Again, use the above priors in doing these calculations.
3) As you're walking to your potions class one afternoon, a noted
prankster from your class challenges you to a game. He shows you two
coins: a fair coin with both a head and tails, and a two-headed coin.
He selects one of the coins at random (equal chance of each being
chosen) and tells you that if you correctly guess which coin it is,
you can have the coin. But wait--there's a catch.
Absolutely. No. Magic. Unluckily for him, you still have Bayes' rule
fresh in your head from the Sorting Hat ordeal.
a) When he flips it, it shows heads. What is the probability that it is the fair coin?
b) Suppose he flips it a second time and again it shows heads. Now what is the probability that it is the fair coin?
Be sure to take into account of the fact that both P(Fair) and P(Heads) need to be updated given the results of (a) above.
c) i. Suppose the coin ever comes up tails. Is it possible for it to be the two headed coin? Why or why not?
ii. Suppose on the third toss (after the first two heads you worked through in (a) and (b) above), the coin comes up tails. Compute P(Fair) with Bayes Rule for this case, and show that it equals 1.
4) It's a shame you spend all your time gambling on funny coins,
because your first essay is due tomorrow, you only have one evening
left to write it, and there are no precocious classmates willing to
write it for you.
As you head into the library, your face drops upon seeing the number
of scrolls you have to sort through to find relevant information.
Luckily, you remember something you learned in CIS140 called
Naive Bayes.
Assume that there are three kinds of scrolls: Relevant (R),
Tangentially Relevant (T), and Irrelevant (I). Assume also that there
are only eight words you will deal with: Witch, Give, Smell, Will,
First, And, The, A, and that capitalization does not matter for our
purposes.
From your previous times in the library, you've estimated the following:
Probabilities of scrolls ever being relevant, tangentially relevant, or irrelevant:
p(R) = .2
p(T) = .3
p(I) = .5
A consultation with a magic pool of water tells you that:
p('Witch' | R) = .7
p('First' | R) = .3
p('Smell' | R) = .9
p('Witch' | T) = .15
p('First' | T) = .2
p('Smell' | T) = .5
p('Witch' | I) = .8
p('First' | I) = .3
p('Smell' | I) = .2
All other words have equal probabilities across relevance type.
a) You open up three scrolls and see the following three sentences, one on each scroll:
i) 'and Will, first give the witch a smell'
ii) 'The first witch will smell and give a'
iii) 'Will a witch give the first smell and'
According to the Naive Bayes classifier, which category is each of
these scrolls most likely to belong to: Relevant, Tangentially
relevant, or Irrelevant?
b) You're starting to get tired from all this reading, so you decide
to read one word per scroll. Assume that you classify a scroll based
on the word 'Witch'.
i) Do you need to know p('Witch')? Why or why not?
ii) If you see a scroll with the word 'Witch', how would the Naive
Bayes classifier classify it (Relevant, Tangentially Relevant, or
Irrelevant)?
c) What assumption does the Naive Bayes classifier make, and what is
the limitation of making that assumption?
|
