Machine Learning / Aprendizagem Automática

2016/2017

1st semester

Announcements

Welcome to the Machine Learning course. This section will contain announcements regarding the operation of the course. Please consult it regularly.

  • The final grades, with the changes resulting from the review of the second exam, have been published.

Presentation

The Machine Learning course is offered by DEEC, in the 1st semester of every year, to the students of the Master's programs on Electrical and Computer Engineering ,on Biomedical Engineering and on Biotechnology, and to the students of the PhD / DEA program on Information Security. It is also open to students of other programs and to voluntary students.

The course's objectives are to transmit the theoretical basis of machine learning, to introduce various types of learning systems, including neural networks, support vector machines, decision trees, and various kinds of unsupervised learning systems, and to give lab practice on the use of several of these systems.

The course is open to foreign students. All essential course materials are in English. The classes will be given in English if there is at least one student who requires it.

Faculty

Note: The e-mail addresses given above are in a form that is not directly usable. To use them, replace " - at - " with "@".

Syllabus

  • Concept of learning. Supervised and unsupervised learning. Reinforcement learning.
  • Learning as optimization. Concept of objective function.
  • Introduction to optimization.
    • The gradient method.
    • Momentum term and adaptive step sizes.
    • Batch and on-line optimization. Ljung's convergence conditions.
  • Supervised learning.
    • The linear learning unit (ADALINE). The LMS algorithm.
    • Multilayer perceptrons (MLPs).
      • Brief reference to Rosenblatt's perceptron.
      • Concept of multilayer perceptron.
      • Training of MLPs. The backpropagation algorithm for the computation of gradients. Batch and on-line training.
    • Generalization.
      • Concept of generalization.
      • Early stopping through cross-validation.
      • Regularization. Some common regularization functionals.
    • Statistical aspects of supervised learning.
      • Regression problems: Estimation of conditional means, medians and percentiles, and of posterior probabilities.
      • The Bayes classifier and the Bayes decision rule.
    • Naive Bayes classifiers. Application to language recognition.
    • Support vector machines.
      • Concept of maximum-margin linear classifier. The associated optimization problem.
      • Non-linearly separable problems. Nonlinear maximum-margin classifiers and kernel methods.
    • Decision trees.
      • Concept of decision tree.
      • Brief introduction to Shannon's entropy.
      • Learning in decision trees. The ID3 method.
  • Unsupervised learning.
    • Clustering and vector quantization. The K-Means algorithm.
    • Estimation of probability densities. The kernel method. The EM algorithm for Gaussian mixture densities.
    • Principal components analysis. Concept of principal components. Estimation of the principal components of a distribution.

Labs
 

Web page

  • Labs and problems web page.
    The web page is accessible through the link above. This page will contain lab assignments, problems for the labs/problems sessions, and related stuff.

Registration for the lab

The registration for the lab will be made through the fenix system. The registration will open on Thursday, September 22 at 17:30. In the beginning of the registration period, the lab (room 5.13, in the 5th floor of the North Tower) will be open, so that the students can use the lab's computers for registration. Prof. Margarida Silveira will be present in the lab to assist with any special registration needs.

To register, log in on the fenix system, open this course's page, and select, in the left-hand menu, "Agrupamentos" (Groups), and then "Labs". Then click on a group that corresponds to the lab shift that you want to register in.

The lab groups are formed by two students each. Students should arrange their own grouping and register in pairs. Please choose a lab partner before the beginning of the registration process.

Students who wish to register individually will have to ask Prof. Margarida Silveira to register them manually. They may have another student assigned to their group, and may find that the work schedules of the two are not very compatible. They will probably have a low priority in choosing the lab shift to which they are assigned.

Students who wish to use previous years' lab grades (from the last two years only) should not register for the lab in the current semester; otherwise their previous grades will be automatically disregarded.

For all matters related to the registration in the lab, please contact Prof. Margarida Silveira.


General information

The labs/problems classes will take place in room 5.13 (also called LSDC1), in the 5th floor of the North Tower. Some classes will be used for solving problems, and other classes will be used for lab work. The Friday shift will only have labs classes: it won't have problems classes.

The labs/problems classes will start in the second week of the semester. The classes of the second and third weeks of the semester will be for solving problems (not for lab work).

The lab assignments will be published, with an advance of about a week, in the Labs and Problems page.

You should prepare your lab work at home, by getting well acquainted with what you will have to do, and by answering the questions that don't require actual lab work. Please note that if you are not well prepared, the the lab class will probably be too short for performing the whole assignment.

The lab reports should be handed in at the end of the corresponding lab sessions.

The reports are made by just filling the appropriate blank spaces in the assignment. You shouldn't exceed the blank spaces that are given. Reports that exceed those spaces will have their grades penalized.

The lab assignments are grading works, and therefore should be done individually by each group. Any sharing of information or joint work among groups will result in a grade of zero.

The lab works are performed using Matlab. If you wish to prepare your work in advance in your own computer, you may want to consider using the Matlab and Simulink Student Version, which is available at a much lower price than the standard version.

Doubts Sessions

To avoid a waste of faculty time, the students who wish to attend a doubts session must send an e-mail message to the corresponding faculty member until 19:00 of the day before the session, and must be present at the time appointed for the session.

Schedule:

  • Prof. Luís B. Almeida - Wednesdays at 11:00.
    Please ask to be announced at the reception desk of the North Tower, to be able to access the 9.th floor.
     
  • Prof. Margarida Silveira - Mondays at 11:30.
    Please go directly to room 5.15 of the North Tower at the appointed time.

This schedule will be maintained after the end of the classes, until the last exam.

Grading

The course's grade is given by the average of the grades of the lab, with a weight of 40%, and of the exam, with a weight of 60%. The grades of both the lab and the exam must be at least 10. The lab grade cannot be more than 2 points above the highest exam grade from the normal grading epoch; this also applies to lab grades carried over from previous years.

Each exam has a duration of three hours. In the exams, the students may only consult a formula sheet (see the indications in the Announcements section, above). Calculators with an alphanumeric keyboard (i.e., with a computer-like keyboard, with individual keys for each of the letters of the alphabet) are not allowed.  Use of phones or of any other communication equipment is not allowed, even for use as calculators or as clocks.

See how to present your responses in the exam.

The students may use, in the current semester, the lab grades from the two previous years, but not beyond that. Students who wish to use previous years' lab grades should not register for the lab in the current semester; otherwise their previous grades will be automatically disregarded.

Exam dates

Please see the fenix system's web page.

Grades

Bibliography

  • Basic bibliography
    The basic bibliography is accessible through the link above. Access requires authentication. The username and password will be given in the course lectures.

  • Complementary bibliography (which is not indispensable for this course):
     
    • Richard O. Duda, Peter E. Hart and David G. Stork, Pattern Classification, 2nd. ed., John Wiley and Sons, 2001 or later.
       
    • Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical Learning, 2nd. ed., Springer, 2009 or later (available online).
       
    • Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, Springer, 2013 or later (available online).
       
    • Tom M. Mitchell, Machine Learning, McGraw-Hill, 1997 or later.
       
    • Jorge Marques, Reconhecimento de Padrões: Métodos Estatísticos e Neuronais, IST press, 1998.
       
    • Simon Haykin, Neural Networks: A Comprehensive Foundation, Prentice-Hall, 1999 or later.
       
    • Martin T. Hagan, Howard B. Demuth, Mark H. Beale and Orlando De Jesús, Neural Network Design, 2nd. ed.; eBook available online; Matlab demo and data files. The eBook is an extended version of the printed book.
       
  • You can find here a set of slides, prepared by Prof. Fernando Silva, corresponding to a previous version of this course. These slides cover a large part of the course, although they don't exactly correspond to the present course. The slides are in Portuguese only. Access requires authentication, as for the basic bibliography.

Problems and exams

Current semester's exams

Previous years' exams

Problems

This page is maintained by Luís Borges de Almeida - Email: luis.almeida - at - lx.it.pt.