Connect with us

Edu News

Unpacking black-box models

Researchers create a mathematical framework to evaluate explanations of machine-learning models and quantify how well people understand them

EP Staff

Published

on

Written by Adam Zewe, MIT News Office

Modern machine-learning models, such as neural networks, are often referred to as “black boxes” because they are so complex that even the researchers who design them can’t fully understand how they make predictions.

To provide some insights, researchers use explanation methods that seek to describe individual model decisions. For example, they may highlight words in a movie review that influenced the model’s decision that the review was positive.

But these explanation methods don’t do any good if humans can’t easily understand them, or even misunderstand them. So, MIT researchers created a mathematical framework to formally quantify and evaluate the understandability of explanations for machine-learning models. This can help pinpoint insights about model behavior that might be missed if the researcher is only evaluating a handful of individual explanations to try to understand the entire model.

“With this framework, we can have a very clear picture of not only what we know about the model from these local explanations, but more importantly what we don’t know about it,” says Yilun Zhou, an electrical engineering and computer science graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and lead author of a paper presenting this framework.

Zhou’s co-authors include Marco Tulio Ribeiro, a senior researcher at Microsoft Research, and senior author Julie Shah, a professor of aeronautics and astronautics and the director of the Interactive Robotics Group in CSAIL. The research will be presented at the Conference of the North American Chapter of the Association for Computational Linguistics.

Understanding local explanations

One way to understand a machine-learning model is to find another model that mimics its predictions but uses transparent reasoning patterns. However, recent neural network models are so complex that this technique usually fails. Instead, researchers resort to using local explanations that focus on individual inputs. Often, these explanations highlight words in the text to signify their importance to one prediction made by the model.

Implicitly, people then generalize these local explanations to overall model behavior. Someone may see that a local explanation method highlighted positive words (like “memorable,” “flawless,” or “charming”) as being the most influential when the model decided a movie review had a positive sentiment. They are then likely to assume that all positive words make positive contributions to a model’s predictions, but that might not always be the case, Zhou says.

The researchers developed a framework, known as ExSum (short for explanation summary), that formalizes those types of claims into rules that can be tested using quantifiable metrics. ExSum evaluates a rule on an entire dataset, rather than just the single instance for which it is constructed.

Researchers use local explanation methods to try and understand how machine learning models make decisions. Even if these explanations are correct, they don’t do any good if humans can’t understand what they mean. MIT researchers have now developed a mathematical framework to quantify and evaluate the understandability of an explanation.
Credits:Image: Courtesy of the researchers

Using a graphical user interface, an individual writes rules that can then be tweaked, tuned, and evaluated. For example, when studying a model that learns to classify movie reviews as positive or negative, one might write a rule that says “negation words have negative saliency,” which means that words like “not,” “no,” and “nothing” contribute negatively to the sentiment of movie reviews.

Using ExSum, the user can see if that rule holds up using three specific metrics: coverage, validity, and sharpness. Coverage measures how broadly applicable the rule is across the entire dataset. Validity highlights the percentage of individual examples that agree with the rule. Sharpness describes how precise the rule is; a highly valid rule could be so generic that it isn’t useful for understanding the model.

Testing assumptions

If a researcher seeks a deeper understanding of how her model is behaving, she can use ExSum to test specific assumptions, Zhou says.

If she suspects her model is discriminative in terms of gender, she could create rules to say that male pronouns have a positive contribution and female pronouns have a negative contribution. If these rules have high validity, it means they are true overall and the model is likely biased.

ExSum can also reveal unexpected information about a model’s behavior. For example, when evaluating the movie review classifier, the researchers were surprised to find that negative words tend to have more pointed and sharper contributions to the model’s decisions than positive words. This could be due to review writers trying to be polite and less blunt when criticizing a film, Zhou explains.

“To really confirm your understanding, you need to evaluate these claims much more rigorously on a lot of instances. This kind of understanding at this fine-grained level, to the best of our knowledge, has never been uncovered in previous works,” he says.

“Going from local explanations to global understanding was a big gap in the literature. ExSum is a good first step at filling that gap,” adds Ribeiro.

Extending the framework

In the future, Zhou hopes to build upon this work by extending the notion of understandability to other criteria and explanation forms, like counterfactual explanations (which indicate how to modify an input to change the model prediction). For now, they focused on feature attribution methods, which describe the individual features a model used to make a decision (like the words in a movie review).

In addition, he wants to further enhance the framework and user interface so people can create rules faster. Writing rules can require hours of human involvement — and some level of human involvement is crucial because humans must ultimately be able to grasp the explanations — but AI assistance could streamline the process.

As he ponders the future of ExSum, Zhou hopes their work highlights a need to shift the way researchers think about machine-learning model explanations.

“Before this work, if you have a correct local explanation, you are done. You have achieved the holy grail of explaining your model. We are proposing this additional dimension of making sure these explanations are understandable. Understandability needs to be another metric for evaluating our explanations,” says Zhou.

This research is supported, in part, by the National Science Foundation.

Business & Economy

Campus 365 provides cloud-based seamless solutions to educational institutions 

With its ERP software and pioneering technological innovation, Campus 365 is disrupting the education sector and making education more effective and accessible

EP Staff

Published

on

Just as technology, and particularly the digital era, has disrupted and improved most major segments of India’s economy, education and training is also undergoing a tech revolution. Education Technology, more commonly known as EdTech, possesses the power to bridge the learning gap thanks to technology’s ability to eliminate geographical barriers. As much as digital adoption is the need of the hour, it is critical to retain a few unparalleled benefits of a traditional classroom setup such as group discussion, personalised support from educators and creating opportunities to work on peer collaborative assignments.

Armed with a mission to empower schools with tools that serve people in the most human, helpful way, Campus 365- an ed-tech start-up incubated by NASSCOM 10000 startup based out of Gurgaon, is providing cloud-based solutions to schools and colleges to manage and track their daily activities seamlessly. With user-friendly mobile apps to help students, parents, and teachers maintain the academic cycle, Campus 365 provides a 360-degree outlook to stakeholders assisting them in making productive decisions to escalate their progress.

“We all know that the introduction of EdTech is transforming the learning process in schools. Initially, technological abilities were not needed for students but these skills are now of much importance. Today, students should have good knowledge of online communication to attend classes, check emails, and submit assignments online and our school ERP gives our students a dedicated platform for this to make them future-ready. We at Campus 365 believe that technology is a powerful instrument that can elevate education to an advanced level, allowing students to learn and collaborate in new ways and we are constantly working towards it” said  Mayank Singh Co- Founder and CEO of Campus 365.

Campus 365 is India’s largest all-in-one app for teachers, schools, and coaching institutes to manage their online and offline classrooms in one place. This online teaching and institute management app helps users create tests & quizzes, share study material & homework, chat with students, take live online classes, record live lectures, teach using digital whiteboard, automate attendance, and much more.

Empowering 1000+ institutes globally, Campus 365’s mission is to enhance student learning by bringing innovation to education. Campus 365 partners with schools and colleges across the country to provide students with a holistic learning experience that makes them future-ready. The company offers a high-quality, end-to-end solution for every stakeholder involved in a child’s learning journey. Campus 365 tackles some of the most critical aspects of School management and reduces school staff workload. By doing so, the company ensures that school teachers nurture the young minds of tomorrow and that the school support staff enjoys using the Campus 365 School ERP engine’s superior apps. Campus 365’s innovative school management software solutions are aimed at providing the best of School ERP features and safety measures to school-going children.

Campus 365’s team is founded with a deep-rooted passion toward delivering value to not just schools, but to parents and teachers as well, who ensure that children are always walking on the path of constant innovation. Today, Campus 365 is used by schools around the world – from kindergarten to higher studies schools and non-profit schools.

Campus 365 has simplified the managing of day-to-day activities such as attendance marking, alerts through mobile applications, fees payment reminders, parent-teacher notifications, and holiday schedules. Every team member in Team Campus 365 has some experience in School and College Management Process and it is also one of the reasons why the company puts concerted efforts into optimising the School and College Management Software. Breaking complex software and making them user-friendly is the top priority of Team Campus 365.

Continue Reading

Edu News

Study Australia successfully concludes the Study Australia Entrepreneurship Challenge

The winners of the SAEC 2022 are – Team Econominds of MITWPU University, Pune

EP Staff

Published

on

The Australian Trade and Investment Commission (Austrade- Australian Government agency) successfully concluded its Study Australia Entrepreneurship Challenge (SAEC-2022) in Delhi on September 23, 2022.

The winners of the SAEC 2022 are – Team Econominds of MITWPU University, Pune. The all-girls team has won a Student Tour to Australia, which will take place in March 2023.

Their theme was Circular Economy and Challenge question was – ‘How might we ensure that the clothing we wear is more sustainable’. The members of the team are Nishita Bhattad, Jasmine Mehta, Samiksha Joshi, Shweta Karthik and Muskan Kashyap.

The award is sponsored by Study Adelaide.

During this tour the winners will get a glimpse of the Australian lifestyle and will get the chance to visit and learn more about Australian education institutions and Australian entrepreneurship ecosystem and start-up culture.

The first runners-up of SAEC 2022 are Team Ekta from NMIMS, Mumbai. They have won a $2000 scholarship each, sponsored by Study New South Wales – which the students will be able to use towards the tuition fee for a full-time postgraduate degree program at any NSW education provider.

The second runners-up of SAEC 2022 are Team Glovis Rise from Lovely Professional University, Punjab who bagged a cash prize of $500 each, sponsored by Study Queensland.

Ms. Melissa Bank, Head of International Education Centre of Excellence, Austrade said, “Australia has always been a preferred destination for Indian students when it comes to higher education. We are seeing an increase in the number of Indian students applying to Australian universities every year. With the re-opening of borders after the pandemic, we are witnessing a constant surge in number of students arriving in Australia. The Australian government is committed to provide them the best-in-class education while also equipping them with necessary skills to facilitate their dream of becoming global citizens. Our goal behind setting up this competition was to help students learn about Australian entrepreneurship eco-system and its institutions. I am glad that all the participants were highly involved with this programme and we have been able to highlight the key differential factor that can make them an entrepreneur.”

The Study Australia Entrepreneurship Challenge was an initiative to engage highly-skilled students from the Indian higher education institutes who participated in a series of hybrid events, which were based on Ted X-meets-Hackathon. The participants formed multidisciplinary teams and participated in a two-week program showcasing the opportunities of Australia’s education sector and the Australian entrepreneurial and start-up ecosystems.

As part of the challenge, the teams created a 3-minute pitch video idea on the following challenge themes – Cybersecurity, Circular Economy, Digital Health, and Creative Industries. The participating teams also had access to mentors from Australian academia and industry who guided them in the development of their video pitches. They got in-depth understanding of their chosen topic during academic spotlights that were delivered by Australian academic experts, as well. The jury panel for the contest included Ms. Melissa Bank, Head of International Education Centre of Excellence, Austrade and Mr Sukhmeet Singh, Co-founder, A2P Energy.

In recent years, Australia has been a leading destination for Indian students seeking to study abroad, and it has progressively emerged as a preferred choice for many. Since the border re-opening in mid-December 2021 until 22 July 2022, over 260,000 student visa holders reached Australia.

Furthermore, the Australian Trade and Investment Commission’s “The Study Australia Industry Experience Program (SAIEP)” was launched for current Indian and other international students at Australian universities to enhance their employability skills.

More information about Study Australia and the support for Indian students is available at https://www.studyaustralia.gov.au/india

Continue Reading

Edu News

University of Essex Online invites applications for PG Cert Global Mental Health and Wellbeing

University of Essex Online courses are delivered by Kaplan Open Learning in partnership with the University of Essex

EP Staff

Published

on

The University of Essex Online is inviting applications for the October intake of the PG Cert Global Mental Health and Wellbeing. The course deals with a wide range of topics, including identifying and processing challenges which impact global mental health, as well as interpreting and understanding ethical policies for an international framework.

This course is designed with a specific focus on the critical perspectives of global mental health policy and practices.  Students will gain the skills and experience required to pursue a career as a mentor, coach, manager, mental health lead or wellbeing adviser. With further training, one could also become a specialist nurse, specialist mental health practitioner or health service manager.

Upon successful completion of the course, students will be awarded a certificate from the University of Essex – one of the UK’s top 40 universities (Complete University Guide 2023).

Course duration and delivery:  This is a 100% online part-time course with an indicative study duration of eight months.

Course content is delivered through a cutting-edge virtual learning environment (VLE), giving students 24/7 access to study materials, such as lecturecasts. This offers a contemporary and accessible way to interact with multimedia content and check understanding via engaging and interactive activities.

Application deadline: 20 October 2022 

Entry requirements: 

·       For the academic entry route, students must have an undergraduate degree from an approved institution, equivalent to a UK Honours degree, or a relevant professional qualification.

·       For the work experience entry route, students must have at least three years of work experience (voluntary or paid) within a relevant field, supported by two appropriate references.

·       As part of the application process, applicants will be asked to take a short aptitude test which is designed to evaluate their ability to handle the intellectual and practical demands of the course. 

·       If English isn’t your first language, your ability should be equivalent to an IELTS (Academic) score of 6.5. If you don’t hold an IELTS or equivalent qualification, the University offers a free online English test.

How to apply:  

Please visit: https://online.essex.ac.uk/apply/

Tuition fee: £3,945

Scholarship:  Merit-based scholarships are available based on prior academic achievements and work experience.

Continue Reading

Trending