作业代写｜INTRODUCTION TO DATA SCIENCE AND SYSTEMS (M) COMPSCI5089

**Answer all 3 questions **

**This examination paper is worth a total of 60 marks.**

**Computational linear algebra and optimisation**

**(a) **Given a collection of *N *documents *D *= *{**D*1*,…,**D**N**}*, your task is to implement a functionality that provides a list of suggested ‘more like this’ documents. With this problem context, answer the following questions.

(i) Explain how would you represent each document *D **∈ **D *as a (real-valued) vector **d**.

What is the dimension of each vector? [2]

(ii) What does the *L*0 norm of a document vector indicate (in plain English) as per your definition of the document vectors in the previous question? [1]

(iii) How would you define the *L**p *distance between two document vectors **d **and **d ***′ *? [2]

(iv) What distance or similarity measure would you use for finding the set of ‘more like this’ documents for a current (given) document vector **d**, and why. [2]

**(b) **The probability distribution function of an *n *dimensional Gaussian is given by

*f*(**x**) = (**x***− **µ*) *T*Σ *−*1 (**x***− **µ*)*, *

where *µ **∈ *R *n *is the mean vector and Σ *∈ *R *n**×**n *is a square and invertible matrix, called the *covariance *matrix. Consider the particular case of *n *= 2. Answer the following questions.

(i) Plot the contours of the following Gaussians. For each contour plot, show the conditional distributions along the two axes.

*µ *= (0*,*0)

Σ = 1 00 1

*µ *= (0*,*0)

Σ = 0*.*5 00 2

*µ *= (0*,*0)

Σ = 0*.*5 0*.*10*.*5 2

*µ *= (0*,*0)

Σ = 0*.*5 0*.*1*−*0*.*5 2

[2](ii) Which one/ones of the above 4 Gaussian distributions can be reduced to a a single dimensional Gaussian with PCA on the covariance matrix without too much loss of information. Note that you do not need to explicitly compute the Eigenvalues. You should rather derive your answer from a visual interpretation of the contour plots.

Clearly explain your answer. [2]

**(c) **With respect to linear regression, answer the following questions.

(i) Derive the expression for stochastic gradient descent for linear regression with the squared loss function. Clearly introduce your notations for the input/output instances,and the parameter vector. [4]

(ii) Explain how linear regression can be extended to polynomial (higher order) regression?

What is the problem of using high degree polynomials for regression? How can that problem be alleviated? [3]

(iii) A common practice in stochastic gradient descent is to use a variable learning rate *α *for the parameter updates *θ**j *(*t*+1) *← **θ**j *(*t*) +*α *(*t*) *∂**L **∂ θ**j **, *where *θ**j *(*t*) denotes the *j **th *component of the parameter vector *θ *at iteration *t*, and *α *(*t*) denotes the value of the learning rate at iteration *t*. Which of the following alternatives of the learning rate update would you prefer (*α *is a constant) and why?

*a*)*α *(*t*) = *α **t **, **b*)*α *(*t*) = *α *+*t**. *[2]

**Probabilities & Bayes rule**

Consider a card game where you have 4 suits (heart, diamonds, clubs and spades) and in each suit the cards 7, 8, 9, 10, Jack (J), Queen (Q), King (K) and Ace (A). In this question we will use the following commonly used terms:

- the
*the pack*: is the set of all cards that have not been drawn yet. - to
*draw*: is to pick a card at random amongst the pack of remaining cards, removing it from the pack.

- the
*hand*: is the set of cards a player has drawn from the pack. - a
*payout*: is the amount of points you get for a given hand. - to
*fold*: is to stop playing and put back your cards in the pack, forfeiting any payout for this game.

**(a) **Assuming that you draw a single card at random from the pack, give the probabilities for the following events

(i) Drawing an Ace?

(ii) Drawing a red card?

(iii) Drawing a diamonds?

(iv) Drawing a royalty figure (Jack, Queen or King)?

(v) Drawing the Ace of spades?

[5]

**(b) **Now assume that you have already drawn the three cards: 10,J,Q. When drawing two more cards from the pack, what is the probability to obtain:

(i) A pair of two cards with the same value (eg, two Jacks).

(ii) Two pairs (eg, two Jacks and two Queens).

(iii) Three of a kind (eg, three Jacks).

(iv) A sequence of 5 cards (eg, 10, J, Q, K, A). Note that the cards can be of any suit, but there cannot be a break in the sequence.

[4]

**(c) **Now let us assume the following payout table for each hand of 5 cards: hand payout sequence of 5 cards 50

three of a kind 30

two pairs 20

one pair 10

anything else 0

As before, you have the cards 10, J, Q in hand.

(i) If you draw two more cards randomly from the deck, what is the expected value of the payout for this hand? [3]

You may also like:

扫描二维码或者

添加微信skygpa

添加微信skygpa