安全に関する取り組みにFRAMと行動分析を併用するアプローチについて発表する。行動とその変容についての科学な取り組みである行動分析学の知見は様々な領域において有用で，それは安全に関する行動にもあてはまる。そのため、行動分析学を現実世界の安全問題に適用し、人々の注意と行動を常に自分や他の人々の日常の安全行動に集中させるプロセスとして、行動分析学にもとづくBBS (Behavior Based Safety)の取り組みがなされてきた。BBSのベースである行動分析学では随伴性にもとづくオペラント条件付けが重要な役割を果たすが、その分析に用いられる随伴性ダイアグラムの作成と分析は属人性が高いものとなっている。本研究ではそのような随伴性とオペラント条件付けのモデル化と分析にFRAMを用い、ることについて発表する。
Systems-Theoretic Process Analysis (STPA) is a safety analysis approach for evaluating the safety of complex systems. Different industries have employed STPA techniques to identify safety concerns during system operation. From our knowledge, the STPA for security does not gain attention from the industry yet. STPA for security is an extension of STPA that proposes to include security concerns into the analysis. However, the manner to investigate security concerns is different from safety. The former concerns to intentionality actions performed while the latter concerns to unintentional actions. We propose an ontology-based technique that extends STPA to improve identification of causal scenarios and associated casual factors, specifically those related to security. The approach assists the systems engineer to conduct safety and security analyses using STPA with a supporting ontology. A tool that implements the proposed ontology was created to support the systems engineer during the safety and security analysis.
AI/IoTシステムの安全性確保のためにまずAIと安全性をめぐる4つの観点から分析を行う。(a) Attack using AI (b) Attack by AI (c) Attack to AI (d) Measure using AI。次に、IoT システムのセキュリティとリスク評価について研究状況を紹介したのちAI機能を持つIoTシステムのリスク評価に関する考察を行う。
タイトル:DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems
発表者：Lei Ma, 助教, 九州大学
概要：Deep Learning (DL) has achieved tremendous success in many cutting-edge applications. However, the state-of-the-art DL systems still suffer from quality issues. While some recent progress has been made on the analysis of feed-forward DL systems, little study has been done on the Recurrent Neural Network (RNN)-based stateful DL systems, which are widely used in audio, natural languages and video processing, etc. In this paper, we initiate the very first step towards the quantitative analysis of RNN-based DL systems. We model RNN as an abstract state transition system to characterize its internal behaviors. Based on the abstract model, we design two trace similarity metrics and five coverage criteria which enable the quantitative analysis of RNNs. We further propose two algorithms powered by the quantitative measures for adversarial sample detection and coverage-guided test generation. We evaluate DeepStellar on four RNN-based systems covering image classification and automated speech recognition. The results demonstrate that the abstract model is useful in capturing the internal behaviors of RNNs, and confirm that (1) the similarity metrics could effectively capture the differences between samples even with very small perturbations (achieving 97% accuracy for detecting adversarial samples) and (2) the coverage criteria are useful in revealing erroneous behaviors (generating three times more adversarial samples than random testing and hundreds times more than the unrolling approach).
タイトル: Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces
発表者：関山太郎, 助教, NII
概要：
We present a method to extract a weighted finite automaton (WFA) from a recurrent neural network (RNN). Our algorithm is based on the WFA learning algorithm by Balle and Mohri, which is in turn an extension of Angluin's classic \lstar algorithm. Our technical novelty is in the use of regression methods for the so-called equivalence queries, thus exploiting the internal state space of an RNN. This way we achieve a quantitative extension of the recent work by Weiss, Goldberg and Yahav that extracts DFAs. Experiments demonstrate that our algorithm's practicality.
「AI/IoTシステムの安全性確保のための考察」
AI/IoTシステムの安全性確保のためにまずAIと安全性をめぐる4つの観点から分析を行う。(a)Attack using AI、(b) Attack by AI、 (c) Attack to AI、 (d) Measure using AI。次に、IoTシステムのセキュリティとリスク評価について研究状況を紹介したのちAI機能を持つIoTシステムのリスク評価に関する考察を行う。パネルディスカッションのパネリスト。
「システム思考で考える複雑システムの安全とSTAMP」
STAMP for complex system safety design based on systems theory
AI・IoT時代の複雑システムの安全は、従来の安全規格の範囲で守ることはできない。故障がなくても事故は起きるということを前提に安全設計をしないといけない。この背景と、そのための方法論の一つとして期待されているSTAMPの考え方を述べる。
※1：FRAM（Functional Resonance Analysis Method：機能共鳴分析手法）「複数の機能の相互インタラクションが、外乱に柔軟に対応する一方でエスカレーションを起こし、安全を脅かし得る」という考えに基づく分析手法
※2：STAMP（System Theoretic Accident Model and Process）「安全/事故はシステムの構成要素の相互作用による創発特性である」という理論に基づく分析手法
Remote attendance will be avaliable and the url will be announced to those who want.
Invited talk
Invited Speaker: Amel Bennaceur, Lecturer, the Open University, UK
Title: Machine Learning Software is Still Software
Abstract:
Machine Learning (ML) is the discipline that studies methods for automatically
inferring models from data. Machine learning has been successfully applied in
many areas of software engineering ranging from behaviour extraction, to
testing, to bug fixing. However, there is comparatively less research on applying
software engineering techniques to designing and implementing machine
learning applications.
Machine learning techniques disrupt the traditional models of software
development and call for quicker, if not immediate, response from requirements
engineering (RE). Indeed, the social underpinning and the increasing reliance on
software systems for every aspect of our life, call for better methods to
understand the impact and implications of software solutions on the wellbeing of
individuals and society as a whole. The intrinsic ability of RE to deal with
conflicts, negotiation, and its traditional focus on tackling those wicked problems
is highly beneficial.
The seminar will review and reflect on the synergies between machine learning
and software engineering. In this seminar, I will introduce the principles of
machine learning, give an overview of some key methods, and present examples
of interaction between software engineering and machine learning. I will also
discuss some open challenges on how machine learning can benefit from
software engineering methods in general and requirements engineering in
particular.
Bio:
Dr. Amel Bennaceur is a Lecturer (Assistant Professor) in Computing at the Open
University, UK. She received her PhD degree in Computer Science from the
University of Paris VI in 2013. Her research interests include dynamic mediator
synthesis for interoperability and collaborative security.
She was part of the Connect and EternalS EU projects that explored synergies
between machine learning and software synthesis.
The results of her work have been published in leading conferences and journals
such as Middleware, ECSA, and IEEE TSE. Bennaceur has been a member of the
program committee of several software engineering conferences including
RE:Next 2016 and ESEC/FSE 2015-NIER. She has been the program co-chair for
ESEC/FSE 2017 Artifact track and SEAMS 2019 and is co-chairing the Poster
Track at RE 2020.
Title: Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems
Abstract:
More and more software practitioners are tackling towards industrial applications of artificial intelligence (AI) systems, especially those based on machine learning (ML). However, many of existing principles and approaches to traditional systems do not work effectively for the system behavior obtained by training not by logical design. In addition, unique kinds of requirements are emerging such as fairness and explainability. To provide clear guidance to understand and tackle these difficulties, we present an analysis on what quality concepts we should evaluate for AI systems. We base our discussion on ISO/IEC 25000 series, known as SQuaRE, and identify how it should be adapted for the unique nature of ML and Ethics guidelines for trustworthy AI from European Commission. We thus provide holistic insights for quality of AI systems by incorporating the ML nature and AI ethics to the traditional software quality concepts.
※1：FRAM（Functional Resonance Analysis Method：機能共鳴分析手法）「複数の機能の相互インタラクションが、外乱に柔軟に対応する一方でエスカレーションを起こし、安全を脅かし得る」という考えに基づく分析手法
※2：STAMP（System Theoretic Accident Model and Process）「安全/事故はシステムの構成要素の相互作用による創発特性である」という理論に基づく分析手法
吉岡実行委員長と金子プログラム委員長
交流会(28日）にて
