Blog

YouTube video

EPFL CIS-RIKEN AIP Joint Seminar #6 20211215

Date and Time: December 15th 6:00pm – 7:00pm(JST)
10:00am-11:00pm(CET)
Venue:Zoom webinar

Language: English

Speaker: Hidetoshi Shimodaira, RIKEN AIP

Title: Selection bias may be adjusted when the sample size is negative in hierarchical clustering, phylogeny, and variable selection

Abstract:
For computing p-values, you should specify hypotheses before looking at data. However, people tend to use datasets twice for hypothesis selection and evaluation, leading to inflated statistical significance and more false positives than expected. Recently, a new statistical method, called selective inference or post-selection inference, has been developed for adjusting this selection bias. On the other hand, we also face biased p-values in multiple testing, although it is a different type of selection bias. In this talk, I present a bootstrap resampling method with a “negative sample size” for adjusting these two types of selection bias. The theory is based on a geometric idea in the data space, which bridges Bayesian posterior probability to the frequentist p-value. Examples are shown for the confidence interval of regression coefficients after model selection and significance levels of trees and edges in hierarchical clustering and phylogenetic inference.

Bio:
Hidetoshi Shimodaira is a professor at Kyoto University and a team leader at RIKEN AIP. He has been working on theory and methods of statistics and machine learning. His multiscale bootstrap method is used in genomics for evaluating the statistical significance of trees and clusters. His “covariate shift” setting for transfer learning is popular in machine learning.

The field name of our research group in Kyoto University has been changed.

Old field name: Mathematical System Theory
New field name: Statistical Intelligence

Our goal is to explore “understanding and thinking” through statistical methods by mathematically studying the underlying methods of artificial intelligence and data science. The new name of the field of our research group is  “statistical intelligence,” meaning data-driven inductive inference that integrates statistics and machine learning.

This seminar introduces the research of the Mathematical Statistics Team.

YouTube video

2021/07/07 15:00-17:00 Online Streaming via Zoom Webinar (registration required)

Mathematical Statistics Team (https://aip.riken.jp/labs/generic_tech/math_stat/) at RIKEN AIP

Speaker 1: Hidetoshi Shimodaira (30 mins)
Title: Statistical Intelligence for Advanced Artificial Intelligence
Abstract: Our goal is to develop a data-driven methodology with statistical inference for artificial intelligence, which may be called “statistical intelligence.” In the first half of the talk, I overview our research topics: (1) Representation learning via graph embedding for multimodal relational data, (2) Valid inference via bootstrap resampling for many hypotheses with selection bias, (3) Statistical estimation of growth mechanism from complex networks. In the second half of the talk, I discuss a generalization of “additive compositionality” of word embedding in natural language processing. I show the computation of distributed representations for logical operations including AND, OR, and NOT, which would be a basis for implementing “advanced thinking” by AI in the future.

Speaker 2: Akifumi Okuno (30mins)
Title: Approximation Capability of Graph Embedding using Siamese Neural Network
Abstract: In this talk, we present our studies on the approximation capability of graph embedding using the Siamese neural network (NN). Whereas a prevailing line of previous works has applied the inner-product similarity (IPS) to the neural network outputs, the overall Siamese NN is limited to approximate only the positive-definite similarities. To overcome the limitation, we propose novel similarities called shifted inner product similarity (SIPS) and weighted inner product similarity (WIPS) for the siamese NN. We theoretically prove and empirically demonstrate their improved approximation capabilities.

Speaker 3: Yoshikazu Terada (30 mins)
Title: Selective inference via multiscale bootstrap and its application
Abstract: We consider a general approach to selective inference for hypothesis testing of the null hypothesis represented as an arbitrarily shaped region in the parameter space of the multivariate normal model. This approach is useful for hierarchical clustering, where confidence levels of clusters are calculated only for those appearing in the dendrogram, subject to heavy selection bias. Our computation is based on a raw confidence measure, called bootstrap probability, which is easily obtained by counting how many times the same cluster appears in bootstrap replicates of the dendrogram. We adjust the bias of the bootstrap probability by utilizing the scaling law in terms of geometric quantities of the region in the abstract parameter space, namely, signed distance and mean curvature. Although this idea has been used for non-selective inference of hierarchical clustering, its selective inference version has not been discussed in the literature. Our bias-corrected p-values are asymptotically second-order accurate in the large sample theory of smooth boundary surfaces of regions, and they are also justified for nonsmooth surfaces such as polyhedral cones. Moreover, the p-values are asymptotically equivalent to those of the iterated bootstrap but with less computation.

Speaker 4: Thong Pham (30 mins)
Title: Some recent progress in modeling preferential attachment of growing complex networks
Abstract: Preferential attachment (PA) is a network growth mechanism commonly invoked to explain the emergence of those heavy-tailed degree distributions characteristic of growing network representations of diverse real-world phenomena. In this talk, I will review some of our recent PA-related works, including a new estimation method for the nonparametric PA function from one single snapshot and a new condition for Bose-Einstein condensation in complex networks.

RIKEN is currently accepting applications for the FY 2021 Special Postdoctoral Researcher (SPDR) Program, from young, creative, independent researchers who will be a powerful force in furthering RIKEN’s research activities.

RIKEN AIP Mathematical Statistics Team is accepting SPDR applicants.

RIKEN website: Call for applications for the position of Special Postdoctoral Researcher (SPDR) for FY 2021

SIPS-351: Journal of Data Science, Statistics and Visualisation Session

Multiview graph embedding as a generalization of canonical correlation analysis 
Hidetoshi Shimodaira, Kyoto University

Talk slide file (PDF)

Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities. Geewook Kim, Akifumi Okuno, Kazuki Fukui, Hidetoshi Shimodaira. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Main track. Pages 5031-5038.

Talk slide file (PDF)

Paper (PDF)

Invited Talk1: Multiscale Bootstrap for Selective Inference with Applications to Model Selection
Hidetoshi Shimodaira, Kyoto University, Japan

talk slide file (PDF)

Several papers are accepted or just written on representation learning for graph embedding and similarity learning. These are great works of students in our group as well as collaborators.

• Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities (arXiv 2019)
• Robust Graph Embedding with Noisy Link Weights (arXiv)(AISTATS 2019 accepted papers) 
• Graph Embedding with Shifted Inner Product Similarity and Its Improved Approximation Capability (arXiv) (AISTATS 2019 accepted papers) 
• A probabilistic framework for multi-view feature learning with many-to-many associations via neural networks (arXiv)  (ICML 2018)

As a related topic, papers on word embedding and image embedding are also accepted or published.

• Segmentation-free compositional n-gram embedding (arXiv)(NAACL-HLT 2019 accepted papers)
• Word-like character n-gram embedding (W-NUT 2018)
• Segmentation-Free Word Embedding for Unsegmented Languages (EMNLP 2017)
• Spectral Graph-Based Method of Multimodal Word Embedding (TextGraphs-11 2017)
• Image and tag retrieval by leveraging image-group links with multi-domain graph embedding (ICIP 2016) 
• Cross-Lingual Word Representations via Spectral Graph Embeddings (ACL 2016)  (CL-Eigenwords Website)

There are also several papers on theory and applications of statistics.

• An information criterion for auxiliary variable selection in incomplete data analysis (Entropy 2019)
• Selective Inference for Testing Trees and Edges in Phylogenetics (arXiv 2019)
• Transitivity vs Preferential Attachment: Determining the Driving Force Behind the Evolution of Scientific Co-Authorship Networks (ICCS 2018)
• Selective inference for the problem of regions via multiscale bootstrap (arXiv 2018)
• PAFit: an R Package for Estimating Preferential Attachment and Node Fitness in Temporal Complex Networks (arXiv 2017) 
• Joint estimation of preferential attachment and node fitness in growing complex networks (Scientific Reports 2016)
• Cross-validation of matching correlation analysis by resampling matching weights (Neural Networks 2016)  (arXiv)

The website for Kyoto University Artificial Intelligence Research Unit (AI Unit) has been made. You can find research teams of PIs in Kyoto University jointly affiliated with RIKEN AIP. There are currently seven such PIs in Kyoto University.

Just after the lab seminar today, we took our group photo.