Jin-Hong Du

Academic Job Interview

2024-10-13T00:00:00-07:00

Screening interview questions

20-40 mins, 1-4 interviewers
Jot down names of interviewers at the start.
Give an answer, then pause. If no follow up - ask if they would like you to expand anything.
Conversation will end with a chance for you to ask questions
- ask 2 questions
- I feel that you have answered everything, and I guess my last question is what would be the next step.

They all tend to have the same structure: you'll meet with probably multiple members of the search committee (2-5), they'll ask 3-5 questions, usually taking turns, then give you the chance to ask one or two questions. The types of questions will vary based on the type of school (mainly for teaching vs research focused positions), but could include:

Why THIS school/department/position?
Describe a course you'd like to teach/what courses are you interested in teaching.
What are your research plans? (For research focused positions, they might want to know about how you plan to get funding. For teaching focused schools, they may want to know about the feasibility of your work there, re: cost of equipment, likely lack of grad students/postdocs, and involvement of undergraduates)
How does your work further diversity, equity, and inclusion? (In the classroom, in your research, etc. I often got this question about inclusive teaching specifically when interviewing for teaching positions).

It tended to be variations on those four specific topics (fit, teaching, research, equity), with random additional ones thrown in. Like in other interviews, I'd keep your answers relatively brief, and signal enthusiasm about the job. Good luck!

Personal

Tell us a bit about yourself.

Thank you so much for the opportunity of being here. My name is Jin-Hong Du. I am currently fourth-year PhD Candidate at Carnegie Mellon University. I am in the Statistics and Machine Learning joint program. My current research interests lie in semiparametric inference, to combine the predictive strength of machine learning and deep learning methods, with valid statistical inference for high-dimensional outcomes. I have also worked on some applied projects, designing flexible deep generative models for single-cell data analysis.

And finally, I have also been working on overparameterized learning with other collaborators, and for me, I am more interested in cross-validation for overaprameterized models, because ….

What's your expectation of being an assistant professor?

Research: I expect to establish a strong research program of statistical methodology and application in the area of single-cell multiomics. I also hope to establish myself in the field of biological science by collaborating with biologists and medical researchers. I aim to secure funding from federal agencies such as the National Science Foundation (NSF) and the National Institutes of Health (NIH) to support my research program.
Teaching and Mentorship: I expect to create an inclusive and engaging learning environment for students, fostering their interest in statistics and machine learning. I look forward to developing and teaching courses in statistics and machine learning, particularly in high-dimensional statistics and applied regression methods. I also hope to mentor students at all levels, guiding them through research projects and helping them develop essential skills for their academic and professional careers.
Service: I expect to be actively involved in service within the department, contributing to committees and collaborative projects. I also hope to engage with the broader academic community through conference organization and journal reviewing.
Professional Development and Collaboration: I look forward to collaborating with colleagues on research projects, sharing my expertise, and learning from others. I also hope to establish interdisciplinary collaboration with members outside the department, contributing to a diverse and vibrant academic environment.

Interest

Why are you interested in this position? / Why do you want this position? / Why are you interested here? What do you know about our institution?

To be honest, I don’t know a lot about the area. What I read makes me think it would be a fine place for me.

What quality make you fit for this position?
How will this position bring you closer to achieving your academic goals?
What ideal colleagues are you looking for? Who do you think that you can collaborate with?

Contribution

Contribution to department/school/university

What can you contribute to our department?
What strengths would you be added to your department?
What can you contribute to our school and university in addition?

Contribution to the field

What do you hope to contribute to your field during your time at this university?
What do you hope to achieve while employed at our institution?

Contribution other than research and teaching

If there is no pressure from research and teaching, what will you do next?

Research

Where is your research heading in the next 5-10 years?

Start with some general points.
- Establish myself in my field of science.
- Emphasize mentoring.
- Securing funding.
- Establishing a productive international scientific network.
- Impact on society.
Specific points.
- Do you know how to get to these goals? Are the goals realistic? What do you need?
- Embed this in the long-term career goal.
- Ideally, immediate goals, 5-year goals, and career goals.
- Explain the importance of collaboration (with people in the department). Well, I can envision something like a research unit on this broader topic that would fit into my 5-year research goal. That is also going be interesting to other research members in your department. These members could be ....
Example: My long-term goal is to achieve .... During my first 5 years here I want to make decisive progress towards the milestone of .... I can engageing collaborations with the other department members because they would help me ... and increase the scope in such and such a way to help me achieve part of these goals and this would also have the added benefit of adding this particular question/approach/theory of ... to the portfolio of the department. To achieve this I will need .... This is important because society really really needs answers to the question of ... and this is why I believe there is a really good chance to get this work funded by funding agency ....

What research resources will you need to be successful?

computation

What is the status of the publications you listed as 'in preparation' or 'under review'?

Tell us about your research interest/ What is your research expertise? Your future research plan/your research vision? Why is your research important in xx field? How will you build and support your research program?

Teaching

What is your teaching experience? / What courses have you taught previously?

While I haven't had the opportunity to teach a full course independently yet, I have gained valuable teaching experience through my roles as a teaching assistant at both Carnegie Mellon University during my PhD and at the University of Chicago during my Master's program.

For instance, I served as a TA for STAT 32940: Multivariate Data Analysis and Visualization at the University of Chicago. In this role, I supported students with diverse needs, particularly in interdisciplinary subjects like statistics. I also proposed new homework problems categorized into different levels of difficulty, which helped students progress through increasingly challenging material.

Additionally, I've been involved in mentoring junior students, guiding them through complex statistical concepts and methods. For example, I've helped students understand and apply clustering algorithms, guiding them in interpreting results meaningfully.

While these experiences aren't full course instruction, they've provided me with a strong foundation in teaching principles and practices. I'm eager to apply these skills and my teaching philosophy to develop and lead full courses in the future, particularly in areas such as high-dimensional statistics, overparameterized learning, generalized linear regression, modern methods in applied statistics, and statistical computing."

This response acknowledges your lack of full course teaching experience while emphasizing your relevant teaching and mentoring experiences. It also shows your enthusiasm and readiness to take on full course responsibilities in the future.

What will be the course you plan to first teach? How would you teach our students?

What can you teach? What would you like to teach?

While I would love to teaching ..., I'm open to filling needs in the department as well.

Tell us how you deal with a classroom of students with different abilities and levels of motivation?

How do you deal with developing a new course?

What is your pedagogical principle? / teaching goal?
Describe an excellent lecture you delivered and how did you prepare for it?
What type of courses would you like to teach?

DEI

Tell me about your commitment to diversity, What will you do to encourage participation and diversity at our school?

My commitment to diversity is deeply rooted in my personal experiences as a first-generation college student and my academic journey. To encourage participation and diversity at the school, I would create an inclusive learning environment that supports diverse student needs, such as providing tailored explanations for students with different backgrounds. I would also actively mentor students from diverse backgrounds on research projects, fostering a supportive environment that helps them develop essential skills for academic and professional success. Additionally, I would integrate DEI principles into my research by focusing on real-world problems that affect underrepresented groups, contributing to the advancement of statistics and science while addressing important societal issues.

Grant

What will be your grant proposal? Which institute? Timeline?

I will try to submit grant proposal to NSF in the fall of the second year after getting a job. This will be submitted to the Statistics Program of the Division of Mathematical Sciences (DMS) in the Directorate for Mathematical and Physical Sciences (MPS) at the National Science Foundation (NSF). At the same time, I will try to find collaborator, who have been successful in getting grants, to submit a grant proposal to the National Institutes of Health (NIH), as an co-investigators. In the third year, after I get more experience on grant writing, I will submit my R01 to NIH and the division will depend on the collaborators and the application we work on.

My questions

Questions that you have about us?

the culture of the department / research center / institution
research themes and areas of focus
the type and amount of teaching duties
how do people get students, what's the quality, what are the numbers, postdocs, etc
Have schools met with their junior faculty to talk about the tenure clock? Have they reached out to provide extra resources? Is there management training? Is there a strong office, a career office or graduate student office to support them in supporting their students? You want to know about resources and core facilities, what their expectations are, how tenure and salary policies are set.
Ask people their personal views of the department, the school, the area, the chair, and then anything important to you personally or professionally. How are the schools in this area? Is this a place that really embraces diversity, that's working hard on issues of social justice?

Others

When are you available to start?

Ideally, I like to start wait until …. But for the right position, I am flexible.

If we give you an offer, would you take it?

I'm very excited about this position and believe it aligns well with my career goals. I would certainly give serious consideration to any offer you extend.

Need to know more information about the offer, such as the salary range, benefits, and job responsibilities before making a final decision, but I'm certainly looking forward to the possibility of joining your department.

One-on-one interview questions

Ask for schedule before the visit day.

The questions to answer:

Do you have some questions on the department?
What do you want to teach?

The questions to ask:

What are you working currently that you are excited about?
teaching load
How many students do you have?

Fill out the time

Dean/chair

non-technical elevator talk
What's your vision for the department?

non evaluated interview

Communication and presentation

Think Fast, Talk Smart: Communication Techniques Youtube

Spontaneous speaking:

Get out of your own way. Dare to be dull.
Reframe the situation as an opportunity.
Slow down and listen.
Tell a story.
1. structure 1: problem/opportunity, solution, benefits.
2. structure 2: what? so what? now what?

Resources

Interviewing for Academic Faculty Positions from NIH: Youtube
Interviewing for Academic Faculty Positions in Science: Youtube
Try to also talk with LLM! 'Can you give me a mock interview?'

Python Handbook

2024-04-30T00:00:00-07:00

Conda

Install mamba:

conda install mamba -n base -c conda-forge

Remove environment:

conda remove --name r_env --all -y

For MacOS, override your default platform osx-arm64 and replace it with osx-64:

CONDA_SUBDIR=osx-64 mamba create -n renv
mamba activate renv
conda config --env --set subdir osx-64

Tensorflow-gpu in conda (Reference):

export XLA_FLAGS=--xla_gpu_cuda_data_dir=$CONDA_PREFIX

Create a conda environment from an environment.yml file:

conda env create -f environment.yml

Setup of R environment

(Reference)

mamba create -n r_env -y

conda activate r_env
conda config --add channels bioconda
conda config --add channels conda-forge
# conda config --set channel_priority strict
# conda config --set channel_priority false

mamba install r-base=4.1.1 -y
mamba install -c conda-forge -c bioconda r-seurat r-seuratdisk r-signac -y
mamba install -c bioconda bioconductor-genomeinfodbdata -y
mamba install -c bioconda bioconductor-genomeinfodbdata --force-reinstall

Jupyter Notebook

Python kernel

Add a new kernel kernel_name from a conda environment env (Reference):

conda activate env
mamba install ipykernel -y
python -m ipykernel install --name kernel_name
conda deactivate

R kernel

(Reference)

Add a new R kernel kernel_name from a conda environment r_env to another conda environment env:

conda activate r_env
mamba install r-irkernel -y

conda activate env
/raid6/Jinhong/anaconda3/envs/r_env/bin/R -e 'IRkernel::installspec(name="kernel_name", displayname="kernel_name")'

Bash commands

Running scripts

nohup python -u foo.py 0 > foo.out 2>&1 &

nohup Rscript foo.R 0 > foo.out 2>&1 &

Python dependencies

Print the versions of Python packages in the current conda environment:

conda list -f "python|numpy|numba|scipy|pandas|scikit-learn|h5py|anndata|scanpy|statsmodels|matplotlib|joblib|cvxpy|tqdm" | awk '{ print $1" | "$2 }'

R dependencies

Linux

System monitoring

System information:

# for basic system info
landscape-sysinfo

# for detailed system info
top
htop

System update:

sudo apt update && sudo apt upgrade -y && sudo reboot

File management

Delete all files with a certain pattern in the current directory and subdirectories (Reference):

find . -type f -name '*.o' -delete
find . -type f -name '*.o' -exec rm {} \;
rm -- **/*.o

Compress all files in the current directory and subdirectories with a certain pattern:

zip -r archive.zip results/* -i "*gears*"

Create a symbolic link for a folder:

ln -s /path/to/original/directory /path/to/link/location/link_name

Sync files from remote server:

rsync -avu --info=progress2 jinhongd@hydra1.stat.cmu.edu:/path /path

# Example: sync all .sh scripts from source to destination
rsync -zarv --include="*/" --include="*.sh" --exclude="*" "$from" "$to" 

User management

On most Linux servers, create a user with:

sudo adduser USERNAME

Set or reset the password:

sudo passwd USERNAME

Give access to a shared folder:

sudo mkdir -p /data/USERNAME
sudo chown -R USERNAME:USERNAME /data/USERNAME
sudo chmod 711 /data
sudo chmod 700 /data/USERNAME

711 on /data allows the user to enter the directory without listing everyone else’s files.

conda/mamba environment for multiple users:

sudo groupadd micromamba
sudo chgrp -R micromamba /data/miniforge3
sudo chmod 770 -R /data/miniforge3

# Each user who wants to use anaconda must be added to that group. Note that you need to log out or otherwise refresh your shell to see your new membership.
sudo adduser USERNAME micromamba

# Last, each user who wants to use conda needs to add it to their path. This can be done following the instructions in the installation doc:
source /data/miniforge3/bin/activate
conda init

Docker

Dependencies: https://stackoverflow.com/questions/76876492/unable-to-install-docker-on-ubuntu-22-04-2

Permission for apt: https://askubuntu.com/questions/908800/what-does-this-apt-error-message-download-is-performed-unsandboxed-as-root

Docker Desktop: https://docs.docker.com/desktop/setup/install/linux/ubuntu/

Enable lingering for the user: This ensures that the user's systemd services, including the rootless Docker daemon, can continue running even after the user logs out. Code

sudo loginctl enable-linger $(whoami)

Replace $(whoami) with the actual username if performing this for a different user.

VPN / Proxy Checking and Node Selection on Linux Server

This server uses mihomo as a Clash-compatible proxy backend.

Default local endpoints:

# HTTP + SOCKS mixed proxy
http://127.0.0.1:7890

# Mihomo external controller API
http://127.0.0.1:9000

The proxy group used for manual node switching is usually:

Proxy

If the group name is different, list all groups first.

1. Check whether mihomo is running

systemctl status mihomo --no-pager

Expected status:

Active: active (running)

Check whether the proxy and controller ports are listening:

ss -lntp | grep -E '7890|9000'

Expected output should include something like:

127.0.0.1:7890
127.0.0.1:9000

If 7890 is missing, the proxy port is not running.

If 9000 is missing, the external controller is not running, and node switching by curl will not work.

2. Check mihomo config path

The systemd service usually starts mihomo with:

/usr/local/bin/mihomo -d /etc/mihomo

So the active config file should be:

/etc/mihomo/config.yaml

Check important fields:

sudo grep -nE 'mixed-port|external-controller|secret|proxies:|proxy-groups:|rules:' /etc/mihomo/config.yaml

The config should include something like:

mixed-port: 7890
external-controller: 127.0.0.1:9000
secret: ""

It should also include:

proxies:
proxy-groups:
rules:

If the config only has something like:

mixed-port: 7890

then mihomo is running with an incomplete config and has no usable nodes.

3. Test the proxy endpoint

Check whether traffic can go through the proxy:

curl -x http://127.0.0.1:7890 https://ipinfo.io

A successful response should show the outbound IP information.

Test Claude API connectivity:

HTTPS_PROXY=http://127.0.0.1:7890 curl -I https://api.anthropic.com

Test OpenAI / Codex connectivity:

HTTPS_PROXY=http://127.0.0.1:7890 curl -I https://api.openai.com

HTTP status codes such as 401, 403, or 404 usually mean the network path is reachable but authentication or endpoint details are missing.

Network-level failures look like:

Connection refused
Could not resolve host
Operation timed out

4. Check whether the controller API works

curl -s http://127.0.0.1:9000/proxies | jq

If this fails with:

Connection refused

then the controller is not listening on port 9000.

Check logs:

sudo journalctl -u mihomo -n 100 --no-pager

Look for messages related to config parsing, controller startup, or listening ports.

5. List all proxy groups

curl -s http://127.0.0.1:9000/proxies | jq -r '.proxies | keys[]'

Typical groups may include:

Proxy
Auto
DIRECT
GLOBAL

The manually switchable group is usually Proxy.

6. Show available nodes in the `Proxy` group

curl -s http://127.0.0.1:9000/proxies | jq -r '.proxies["Proxy"].all[]'

Show the currently selected node:

curl -s http://127.0.0.1:9000/proxies | jq -r '.proxies["Proxy"].now'

If this returns null, the group name may not be Proxy.

List all groups again:

curl -s http://127.0.0.1:9000/proxies | jq -r '.proxies | keys[]'

Then replace Proxy with the correct group name.

7. Manually switch node

Switch the Proxy group to Auto:

curl -X PUT 'http://127.0.0.1:9000/proxies/Proxy' \
  -H 'Content-Type: application/json' \
  --data '{"name":"Auto"}'

Switch to a specific node:

curl -X PUT 'http://127.0.0.1:9000/proxies/Proxy' \
  -H 'Content-Type: application/json' \
  --data '{"name":"NODE_NAME_HERE"}'

Example:

curl -X PUT 'http://127.0.0.1:9000/proxies/Proxy' \
  -H 'Content-Type: application/json' \
  --data '{"name":"日本-PRO-FW-JP1-流量倍率:0.2"}'

Confirm the selected node:

curl -s http://127.0.0.1:9000/proxies | jq -r '.proxies["Proxy"].now'

8. Useful helper script: `switch-node`

Create a command-line helper:

sudo tee /usr/local/bin/switch-node > /dev/null <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

API="http://127.0.0.1:9000"
GROUP="${GROUP:-Proxy}"

if [ $# -eq 0 ]; then
  echo "Current node:"
  curl -s "$API/proxies" | jq -r ".proxies[\"$GROUP\"].now"

  echo
  echo "Available nodes:"
  curl -s "$API/proxies" | jq -r ".proxies[\"$GROUP\"].all[]"
  exit 0
fi

NODE="$*"

curl -s -X PUT "$API/proxies/$GROUP" \
  -H 'Content-Type: application/json' \
  --data "{\"name\":\"$NODE\"}" >/dev/null

echo "Switched to:"
curl -s "$API/proxies" | jq -r ".proxies[\"$GROUP\"].now"
EOF

sudo chmod +x /usr/local/bin/switch-node

Usage:

# Show current node and available nodes
switch-node

# Switch to Auto
switch-node Auto

# Switch to a specific node
switch-node 日本-PRO-FW-JP1-流量倍率:0.2

If the proxy group is not named Proxy, set GROUP manually:

GROUP='YourGroupName' switch-node
GROUP='YourGroupName' switch-node Auto

9. Enable proxy variables for Claude / Codex / CLI tools

For the current shell:

export HTTP_PROXY=http://127.0.0.1:7890
export HTTPS_PROXY=http://127.0.0.1:7890
export ALL_PROXY=http://127.0.0.1:7890

export http_proxy=http://127.0.0.1:7890
export https_proxy=http://127.0.0.1:7890
export all_proxy=http://127.0.0.1:7890

export NO_PROXY=localhost,127.0.0.1,::1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
export no_proxy=$NO_PROXY

For all SSH users, add these variables to:

/etc/profile.d/proxy.sh

Example:

sudo tee /etc/profile.d/proxy.sh > /dev/null <<'EOF'
export HTTP_PROXY=http://127.0.0.1:7890
export HTTPS_PROXY=http://127.0.0.1:7890
export ALL_PROXY=http://127.0.0.1:7890

export http_proxy=http://127.0.0.1:7890
export https_proxy=http://127.0.0.1:7890
export all_proxy=http://127.0.0.1:7890

export NO_PROXY=localhost,127.0.0.1,::1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
export no_proxy=$NO_PROXY
EOF

After re-login:

env | grep -i proxy

11. Minimal daily workflow

# Check service
systemctl status mihomo --no-pager

# Check ports
ss -lntp | grep -E '7890|9000'

# List available nodes
switch-node

# Switch node
switch-node Auto

# Test outbound IP
curl -x http://127.0.0.1:7890 https://ipinfo.io

# Test Claude
HTTPS_PROXY=http://127.0.0.1:7890 curl -I https://api.anthropic.com

# Test OpenAI / Codex
HTTPS_PROXY=http://127.0.0.1:7890 curl -I https://api.openai.com


# If config changes, restart mihomo
sudo systemctl restart mihomo

SOCKS5-based VPN on Linux

Suppose a VPN/proxy client, such as a Trojan client, provides a local SOCKS5 proxy at:

127.0.0.1:1080

Test whether the SOCKS5 proxy works:

curl -v --socks5-hostname 127.0.0.1:1080 https://ifconfig.me
curl -v --socks5-hostname 127.0.0.1:1080 https://api.anthropic.com

Use --socks5-hostname rather than --socks5 so that DNS resolution is also sent through the proxy.

Some command-line tools do not support SOCKS5 proxies directly. For example, Claude Code respects HTTP_PROXY and HTTPS_PROXY, but does not directly support SOCKS5 proxies. If Claude Code shows errors such as ECONNRESET, bridge the SOCKS5 proxy to a local HTTP proxy using privoxy.

Install privoxy:

sudo apt update
sudo apt install privoxy

Edit the config:

sudo nano /etc/privoxy/config

Add the following lines:

listen-address 127.0.0.1:8118
forward-socks5t / 127.0.0.1:1080 .

Restart privoxy:

sudo systemctl restart privoxy

Then export HTTP proxy variables:

unset ALL_PROXY all_proxy
unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxy

export HTTP_PROXY=http://127.0.0.1:8118
export HTTPS_PROXY=http://127.0.0.1:8118
export NO_PROXY=localhost,127.0.0.1,::1

Test the HTTP proxy bridge:

curl -v --proxy http://127.0.0.1:8118 https://api.anthropic.com
curl -v --proxy http://127.0.0.1:8118 https://claude.ai

Run Claude Code:

claude

If it still fails, inspect the proxy-related environment variables:

env | grep -i proxy

A common bad setup is:

HTTPS_PROXY=socks5://127.0.0.1:1080
HTTP_PROXY=socks5://127.0.0.1:1080

For Claude Code, use the HTTP proxy bridge instead:

HTTPS_PROXY=http://127.0.0.1:8118
HTTP_PROXY=http://127.0.0.1:8118

For browsers, make sure SOCKS5 uses remote DNS. In Firefox, set:

network.proxy.socks_remote_dns = true

For Chrome or Chromium, disabling QUIC may help if HTTPS traffic behaves inconsistently:

chrome://flags/#enable-quic

LaTeX on Linux

sudo apt install texlive-full texlive-latex-recommended

Slurm

Check GPU usage on the node n01:

ssh n01 'nvidia-smi -l 1'

Check the status of the jobs:

# all jobs
squeue -u jinhongd

# pending jobs
squeue --start -u jinhongd

Check the CPU and memory usage of a job:

# for running jobs
sacct -e
sacct --format="CPUTime,MaxRSS"
sacct -a -j <job_id> --format=user%10,jobname%10,node%10,start%10,end%10,elapsed%10,MaxRS

# for finished jobs
seff <job_id>

LLM

claude --dangerously-skip-permissions

codex --dangerously-bypass-approvals-and-sandbox

codex alias

mkdir -p ~/.codex-jing

# Login personal
CODEX_HOME=~/.codex-jing codex login

alias codex-jing='CODEX_HOME=$HOME/.codex-jing codex'


# or add to ~/.bashrc
export CODEX_HOME_WORK="$HOME/.codex-jing"
source ~/.bashrc

Divergence Measures

2022-04-16T00:00:00-07:00

This post summarizes useful results on divergence measurere.

Divergence Measures

Below we use $\mathbb{P}$ and $\mathbb{Q}$ to denote two probability measures. When they are absolutely continuous with respect to the Lebesgue measure $\lambda$, the Radon–Nikodym derivatives $\frac{\mathrm{d}\mathbb{P}}{\mathrm{d}\lambda}$ and $\frac{\mathrm{d}\mathbb{Q}}{\mathrm{d}\lambda}$ reduce to its density function $p$ and $q$. Thus, the following definition of divergence measures can be expressed in terms of density functions provided that the density functions exist. Furthermore, there is no loss of generality in assuming the existence of densities, since any pair of distributions have densities with respect to the base measure $\lambda=\frac{1}{2}(\mathbb{P}+\mathbb{Q})$.

Total variation distance

The total variation (TV) distance is defined as follows: $$TV(\mathbb{P},\mathbb{Q})= \sup\limits_{A\subseteq\Omega}|\mathbb{P}(A)-\mathbb{Q}(A)|.$$ This is a very strong notion of distance. In particular, if the TV is small then the probability of any event under the two distributions must be close.

If the domain $\Omega$ is discrete and suppose we have two distributions $\mathbb{P}$ and $\mathbb{Q}$ with probability mass functions $p$ and $q$, then $$\begin{align*} TV(\mathbb{P},\mathbb{Q})&=1-\sum_{x \in \Omega} \min \{p(x), q(x)\} \\ &= \sum_{{x \in \Omega: p(x) \geq q(x)}}[p(x)-q(x)]\\ &= \frac{1}{2} \sum_{x \in \Omega}|p(x)-q(x)| \end{align*}$$ The Total Variation distance also belongs to a popular class of distances between probability distributions. It is an integral probability metric (IPM) (other popular examples of IPMs include the Wasserstein distance). The TV distance can also be written as: $$TV(\mathbb{P},\mathbb{Q})=\frac{1}{2} \sup_{||f||_{\infty} \leq 1}\left|\mathbb{E}_{X \sim \mathbb{P}}[f(X)]-\mathbb{E}_{Y \sim \mathbb{Q}} [f(Y)]\right|$$

Squared Hellinger distance

The squared Helligher distance is defined as $$H^2(\mathbb{P},\mathbb{Q})=\frac{1}{2}\int\left(\sqrt{\frac{\mathrm{d}\mathbb{P}}{\mathrm{d}\lambda}}- \sqrt{\frac{\mathrm{d}\mathbb{Q}}{\mathrm{d}\lambda}}\right)^2\mathrm{d}\lambda$$ We can also define it for corresponding density function (or probability mass functions) $p$ and $q$ as $$ H^{2}(p,q)={\frac {1}{2}}\int \left({\sqrt {p(x)}}-{\sqrt {q(x)}}\right)^{2}\mathrm{d}x=1-\int {\sqrt {p(x)q(x)}}\mathrm{d}x.$$ The Helligher distance and the squared Helligher distance are bounded by 0 and 1: $$0\leq H^2(\mathbb{P},\mathbb{Q})\leq H(\mathbb{P},\mathbb{Q})\leq 1.$$

Kullback–Leibler divergence

$$KL(\mathbb{P}, \mathbb{Q}) = \mathbb{E}_{\mathbb{P}}\left[\log\frac{\mathrm{d}\mathbb{P}}{\mathrm{d}\mathbb{Q}}\right]$$

$$KL(p, q) = - \int p(x) \log q(x) dx + \int p(x) \log p(x) dx$$

Chi-square divergence

Properties

Comparison

$$H^2(\mathbb{P},\mathbb{Q})\leq TV(\mathbb{P},\mathbb{Q})\leq \sqrt{2} H(\mathbb{P},\mathbb{Q}) \lesssim\sqrt{KL(\mathbb{P},\mathbb{Q})} \lesssim \sqrt{\chi^2(\mathbb{P},\mathbb{Q})}$$

Tenserization

If $X_1,\ldots,X_n\overset{iid}{\sim}\mathbb{P}$, then they are associated with the product measure $\mathbb{P}^n$. What is the relationship between divergence of the produce measure and the individual measure? It turns out that for the KL divergence and Hellinger distance we have $$\begin{align*} KL(\mathbb{P}^n, \mathbb{Q}^n) &= n KL(\mathbb{P}, \mathbb{Q})\\ H^2(\mathbb{P}^n,\mathbb{Q})^n &\leq n H^2(\mathbb{P},\mathbb{Q}). \end{align*}$$

Example

Suppose $p$ and $q$ are the density function of $\mathcal{N}(\mu_1,\sigma_1^2)$ and $\mathcal{N}(\mu_2,\sigma_2^2)$ respectively, then $$\begin{align} KL(p, q) &= - \int p(x) \log q(x) dx + \int p(x) \log p(x) dx\\ &=\frac{1}{2} \log (2 \pi \sigma_2^2) + \frac{\sigma_1^2 + (\mu_1 - \mu_2)^2}{2 \sigma_2^2} - \frac{1}{2} (1 + \log 2 \pi \sigma_1^2)\\ &= \log \frac{\sigma_2}{\sigma_1} + \frac{\sigma_1^2 + (\mu_1 - \mu_2)^2}{2 \sigma_2^2} - \frac{1}{2} \end{align}$$

Norm Inequality

2022-04-16T00:00:00-07:00

This post summarizes some useful inequalities between different norms.

Norm Equivalence

On any finite-dimensional vector space over $\mathbb{R}$ or $\mathbb{C}$, all norms are equivalent. This means that for any two norms $\Vert\cdot\Vert_a$ and $\Vert\cdot\Vert_b$, there exists $c_1\geq c_2>0$ such that $$c_1\Vert\cdot\Vert_b \leq \Vert\cdot\Vert_a \leq c_2\Vert\cdot\Vert_b $$ holds for any element in the space.

Useful Inequalities

Domain	Inequality
Vector $x\in\mathbb{R}^n$ with $s=\Vert x\Vert_0\leq n$	$\Vert x\Vert_{\infty}\leq \Vert x\Vert_{2} \leq \Vert x\Vert_{1} \leq \sqrt{s}\Vert x\Vert_{2} \leq s\Vert x\Vert_{\infty}$
Spectral domain of Matrix $X\in\mathbb{R}^{n\times d}$ with $r=\text{rank}(X)\leq \min\{n,d\}$	$\Vert X\Vert_{op}\leq \Vert X\Vert_{F} \leq \Vert X\Vert_{nuc} \leq \sqrt{r}\Vert X\Vert_{F} \leq r\Vert X\Vert_{op}$
Matrix $X\in\mathbb{R}^{n\times d}$	$\frac{1}{\sqrt{d}}\Vert X\Vert_\infty\leq\Vert X\Vert_{op}\leq \sqrt{n} \Vert X\Vert_\infty$ $\frac{1}{\sqrt{n}}\Vert X\Vert_1\leq \Vert X\Vert_{2,\infty} \leq \Vert X\Vert_{op}\leq \sqrt{d} \Vert X\Vert_{2,\infty} \leq \sqrt{d}\Vert X\Vert_1$

Note the matrix norms in the spectral domain have similar behaviors as the corresponding vector norms.

Miscellanies

$\Vert X\Vert_{1} = \max_{1\leq j\leq d} \Vert X_j \Vert_1$
$\Vert X\Vert_{2,\infty} = \max_{1\leq j\leq d} \Vert X_j \Vert_2$
$\Vert X \Vert_{F} = ({\rm tr}(X^{\top}X))^{1/2}$
$\Vert X \Vert_{nuc} = \Vert X \Vert_{\ast} = \Vert X \Vert_{\rm tr} = {\rm tr}((X^{\top}X)^{1/2})$

$|{\rm tr}(AB)| \leq \Vert A \Vert_{\rm op} \cdot \Vert B \Vert_{\rm tr} \leq \sqrt{\rm rank}(B) \Vert A \Vert_{\rm op} \cdot \Vert B \Vert_{F}$

For positive semi-definite matrix $A$ and arbitrary $B$, we have $\lambda_n(B),{\rm tr}(A)\leq {\rm tr}(AB)\leq\lambda_1(B),{\rm tr}(A)$

Reproducing Kernel Hilbert Spaces

2022-03-22T00:00:00-07:00

This post summarizes reproducing kernel Hilbert spaces and mainly follows the lecture slides by D. Sejdinovic, A. Gretton.

We will mainly be concerned about the field $\mathbb{R}$.

Elementary Spaces

The following graph from the slides illustrates the reproducing kernel Hilbert spaces (RHKS) pretty well.

We summarize some important spaces in the following table.

Space	Description	Characterization
vector space
normed vector space	a vector space endowed with a norm, which is also a metric space	distance
inner product vector space	a vector space endowed with an inner product	angles and orthogonality
complete space	every Cauchy sequence converges in the space	completeness
Banach space	complete normed vector space
Hilbert space	complete inner product vector space

For the following content, we will restrict ourselves to the norm vector space, in which some nice properties can be established.

Linear Operators

A linear operator is a function $A:\mathcal{F}\rightarrow\mathcal{G}$ such that $$A(a_1 f_1+a_2f_2)=a_1 A(f_1)+a_2A(f_2),\qquad\forall\ a_1,a_2\in\mathbb{R},f_1,f_2\in\mathcal{F},$$ where $\mathcal{F}$ and $\mathcal{G}$ are two vector spaces over $\mathbb{R}$. When $\mathcal{G}=\mathbb{R}$, the operators are called functionals.

If $\mathcal{F}$ and $\mathcal{G}$ are two normed vector spaces over $\mathbb{R}$, then the operator norm of a linear operator $A$ is defined as $$||A||=\sup\limits_{f\in\mathcal{F},||f||_{\mathcal{F}}\leq 1}||Af||_{\mathcal{G}}.$$ In this case, a bounded linear operator is one with finite operator norm. bounded operator $\neq$ bounded function.

Continuity

If $\mathcal{F}$ and $\mathcal{G}$ are two normed vector spaces over $\mathbb{R}$, then $A:\mathcal{F}\rightarrow\mathcal{G}$ is continuous at $f_0\in\mathcal{F}$ if $\forall\ \epsilon>0$, $\exists\ \delta>0$ such that $||f-f_0||_{\mathcal{F}}<\delta\Rightarrow||Af-Af_0||_{\mathcal{G}}<\epsilon$. If $A$ is continous at any $f\in\mathcal{F}$, then $A$ is continuous on $\mathcal{F}$.

Let $(\mathcal{F},||\cdot||_{\mathcal{F}})$ and $(\mathcal{G},||\cdot||_{\mathcal{G}})$ be normed linear spaces. If $A$ is a linear operator, then the following three conditions are equivalent:

$A$ is a bounded operator.
$A$ is continuous on $\mathcal{F}$.
$A$ is continuous at one point of $\mathcal{F}$.

Duality

If $\mathcal{F}$ is a normed space, then the space $\mathcal{F}^{\ast}$ of continuous linear functionals $A:\mathcal{F} \rightarrow \mathbb{R}$ is called the topological dual space of $\mathcal{F}$.

(Riesz representation) In a Hilbert space $\mathcal{F}$, for every continous linear functional $A \in \mathcal{F}^{\ast}$, there exists a unique $g \in \mathcal{F}$, such that $Af=\langle f,g\rangle.$

Riesz representation gives an isomorphism $g\mapsto\langle \cdot, g\rangle_{\mathcal{F}}$ between $\mathcal{F}$ and $\mathcal{F}^{\ast}$ which preserves the inner product in the two spaces. This implies that dual space of a Hilbert space is also a Hilbert space.

Isometric isomorphism

Two Hilbert spaces $\mathcal{H}_1$ and $\mathcal{H}_2$ is isometrically isomorphic if there exists a linear bijective map $U:\mathcal{H}_1\rightarrow\mathcal{H}_2$ that preserves the inner product, i.e., $\langle h_1,h_2\rangle_{\mathcal{H}_1}=\langle Uh_1,Uh_2\rangle_{\mathcal{H}_1}$.

(Isometric isomorphism between Hilbert spaces and sequence spaces) Every Hilbert space has an orthonormal basis and isometrically isomorphic to $l^2(A)$ for some set $A$.

When the Hilbert space is separable, then $A$ can be taken as $\mathbb{N}$.

RKHS

Evaluation functional $\delta_x:f\mapsto f(x)$ can be discontinuous in a Hilbert space $\mathcal{H}$ of functions $\mathcal{X}\rightarrow\mathbb{R}$.

A Hilbert space $\mathcal{H}$ of functions $\mathcal{X}\rightarrow\mathbb{R}$ is said to be a reproducing kernel Hilbert space (RKHS) if $\delta_x$ is an continuous functional for all $x\in\mathcal{X}$.

Note that we have restrict the objects in the space to be functionals $\mathcal{X}\rightarrow\mathbb{R}$.

Norm convergence in a RKHS implies pointwise convergence. If $\lim_{n\rightarrow\infty}||f_n-f||_{\mathcal{H}}=0$, then $\lim_{n\rightarrow\infty}f_n(x)=f(x)$, for all $x\in\mathcal{X}$.

Reproducing kernel

Inner product between features

Positive definite function

Differential Theory

2021-12-21T00:00:00-08:00

This post summarizes differential theory on convex functions, including directional derivatives, subgradients, and Legendre transformation.

Characterization of Smoothness

Directional Derivatives

The directional derivative of $f$ at $x$ in the direction $d$ is defined as $$\nabla_d f(x)=\lim\limits_{t\rightarrow 0+}\frac{f(x+td)-f(x)}{t}$$

If $f$ is finite-value and convex, then the directional derivative for fixed $x$ is finite sublinear with respect to $d$.

Subgradients

The subdifferential $\partial f (x)$ of $f$ at $x$ is the nonempty compact convex set of $\mathbb{R}^n$ whose support function is the directional derivative, i.e. $$\partial f(x)=\{s\in\mathbb{R}^n\mid \langle s, d\rangle \leq \nabla_d f(x),\ \forall\ d\in\mathbb{R}^n\}$$ Any vector $s\in \partial f(x)$ is called a subgradient of $x$.

The subgradient can also be defined via the subgradient inequality. We say $s$ is a subgradient of $f$ at $x$ if $$f(y)\geq f(x)+ \langle s, y-x\rangle,\quad \forall\ z\in\mathbb{R}^n.$$

Property

(Multiplicability) If $f$ is a proper convex function on $\mathbb{R}^n$, then $\partial(\lambda f)=\lambda \partial f,\ \forall\ x\in\mathbb{R}^n,\lambda>0.$
(Additivity) If $f_1$ and $f_2$\text{ri}( are proper convex functions on $\mathbb{R}^n$, then $\partial(f_1+f_2)\supseteq \partial f_1+ \partial f_2,\ \forall\ x\in\mathbb{R}^n$. If the convex sets $\text{ri}(\text{dom} f_1)\cap \text{ri}(\text{dom} f_1)\neq\varnothing$, then $\partial(f_1+f_2)= \partial f_1+ \partial f_2,\ \forall\ x\in\mathbb{R}^n$.
(Affine Composition) Let $f(x)=h(Ax)$ where $h$ is a proper convex function on $\mathbb{R}^m$ and $A$ is a linear transform from $\mathbb{R}^n$ to $\mathbb{R}^m$. Then $\partial f\supset A^{\ast}\partial h(Ax),\ \forall\ x\in\mathbb{R}^n$. If $\text{Im}(A)\cap \text{ri}(\text{dom}(h))\neq\varnothing$ or if $h$ is polyhedral and $\text{Im}(A)\cap \text{dom}(h)\neq\varnothing$, then $\partial f= A^{\ast}\partial h(Ax),\ \forall\ x\in\mathbb{R}^n$.

Essential Smoothness and Essential Strictly Convexity

A proper convex function $f$ on $\mathbb{R}^n$ is essentially smooth if it satisfies the following conditions for $C=\text{dom}(f)$:

$C\neq\varnothing$;
$f$ is differentiable on $C$;
$\lim\limits_{\substack{x\rightarrow x_0\\ x\in C}}|\nabla f(x)|=+\infty$ for all $x_0\in\text{bd}(f)$, which is equivalent to $\partial f(x_0)=\varnothing$ for all $x_0\in\text{bd}(f)$.

The subtle difference between essential smoothness and smoothness lies in the boundary behavior of the gradient. A smooth convex function on $\mathbb{R}^n$ is in particular essentially smooth.

(Essential Smoothness and Subgradient) If $f$ is a closed proper convex function, then $f$ is essentially smooth if and only if $$\partial f=\begin{cases}{\nabla f}&\quad x\in\text{int}(\text{dom}(f)),\\\varnothing&,\quad x\not\in\text{int}(\text{dom}(f)).\end{cases}$$

A proper convex function $f$ on $\mathbb{R}^n$ is essentially smooth if $f$ is strictlt convex on any convex subset of $\text{dom}(\partial f)=\{x\mid \partial f(x)\neq \varnothing\}$.

Smoothness and Strictly Convexity

The duality between essential smoothness and esstential strictly convexity can be characterized as follows.

(Essential Smoothness and Essential Strictly Convexity) A closed proper convex function $f$ is esstentially strictly convex if and only if its convex conjugate $f^{\ast}$ is essentially smooth.

(Subgradient) A closed proper convex function $f$ has one-to-one subgradient mappings if and only if $f$ and $f^{\ast}$ are essentially smooth; if and only if $f$ is strictly convex on $\text{int}(\text{dom}(f))$ and is essentially smooth.

The Legendre Transformation

Note that from the definition of convex conjugate $$ \begin{align*} f^{\ast}(u) &= \sup\limits_{x\in\mathbb{R}^n} x^{\top}u - f(x) \end{align*}$$ If $f$ is differentiable (on its domain), we take the derivative and set to zero: $$u-\nabla f(x)=0.$$ If $\nabla f$ is a one-to-one mapping, then $x=[\nabla f]^{-1}(u)$ obtains the supremum. Thus, $$f^{\ast}(u) = \langle [\nabla f]^{-1}(u), u \rangle- f([\nabla f]^{-1}(u)),$$ which is called the Legendre conjugate.

Duality

2021-12-20T00:00:00-08:00

This post summarizes duality.

Separation Theorem

Convex Conjugates

Primal-Dual Formulation

Convex Sets and Convex Functions

2021-12-17T00:00:00-08:00

This post summarizes convex sets and convex functions, and their properties.

Convex Sets

Convex Functions

(Convexity) A function $f$ is convex if $$f(tx+(1-t)y)\leq tf(x)+(1-t)f(y),$$ for all $x,y\in dom(f)$ and $t\in(0,1)$.

(Strict Convexity) A function $f$ is strictly convex if $$f(tx+(1-t)y)< tf(x)+(1-t)f(y),$$ for all $x,y\in dom(f)$ and $t\in(0,1)$.

(Strong Convexity) A function $f$ is $m$-strongly convex if $f(x)-\frac{m}{2}\|x\|_2^2$ is convex.

Strong convexity implies strict convexity, which implies convexity.

If a convex function has a local minimum $x^{\ast}$, then $x^{\ast}$ is also a global minimum.
If a strictly convex function has a local/global minimum $x^{\ast}$, then $x^{\ast}$ is the unique global minimum.

A proper convex function is an extended real-valued convex function with a non-empty domain, that never takes on the value $-\infty$ and also is not identically equal to $+\infty $.

Convex Functions over $K$

Causal Inference

2021-12-07T00:00:00-08:00

This post summarizes causal inference.

The Potential Outcomes Framework

Suppose we have two random variables $X$ and $Y$, where $Y$ is a binary outcome. We are interested in determining whether $X$ causes $Y$. However, for the $i$th unit/sample, we can also observe $Y_i^{obs}$, which is either $Y_i(1)$ or $Y_i(0)$. We aim to use $(X_i,Y_i^{obs})$'s to estimate some quantities related to the causal effect of treatment.

The causal estimands include

The unit level causal effects $Y_i(1)-Y_i(0)$, $Y_i(1)/Y_i(0)$, and etc.
The average treatment effect: $\hat{\tau}=\frac{1}{n}\sum\limits_{i=1}^n (Y_i(1)-Y_i(0))$ for estimating $\tau=\mathbb{E}[Y(1)-Y(0)]$.
The average treatment effect over sub-populations: $\hat{\tau}_S=\frac{1}{|S|}\sum\limits_{i=1}^n (Y_i(1)-Y_i(0))\mathbf{1}\{i\in S\}$.

Suppose that if there is a causal link between a binary treatment $W$ and an outcome $Y$. For a particular unit, if $W_i= 1$ we say that the unit is treated and if $W_i = 0$ then the unit is in the control group. What we observe is $$Y_i^{obs}=W_iY_i(1)+(1-W_i)Y_i(0).$$ We can estimate the association $$\alpha=\mathbb{E}[Y(1)\mid W=1]-\mathbb{E}[Y(0)\mid W=0]$$ by an estimator $$\hat{\alpha}=\frac{1}{|T|}\sum\limits_{i\in T}Y_i^{obs}-\frac{1}{|C|}\sum\limits_{i\in C}Y_i^{obs}$$ where $T=\{i:W_i=1\}$ and $C=\{i:W_i=0\}$.

In general, the association is not equal to the average causal effect, i.e., $\alpha\neq\tau$, which is caused by the selection bias. However, if we consider randomized controlled trials (RCT) by enforcing the condition that $$W\perp\!\!\!\!\perp (Y(1),Y(0)),$$ then we have $\alpha=\tau$.

Confounding

Identification of $\tau$

For most studies, we do not have randomly trials and can only have observation data. To make causal inference from observational data possible in such case, we need to assume no unmeasured confounding or no selection on observables or ignorability. Formally, we assume $$W\perp\!\!\!\!\perp (Y(1),Y(0))\mid X,$$ where $X$ is some covariate.

Note that under the previous assumption, we have $$\begin{align*} \tau & = \mathbb{E}[Y(1)-Y(0)]\\ &= \mathbb{E}[\mathbb{E}[Y(1)-Y(0)\mid X]]\\ &= \mathbb{E}[\mathbb{E}[Y(1)\mid X,W=1]]-\mathbb{E}[\mathbb{E}[Y(0)\mid X,W=0]]\\ &= \mathbb{E}[\mathbb{E}[Y^{obs}\mid X,W=1]]-\mathbb{E}[\mathbb{E}[Y^{obs}\mid X,W=0]]. \end{align*}$$ Thus, we are able to estimate $\tau$ from the observed data $(X,W,Y^{obs})$ instead of the original data $(X,W,Y(1),Y(0))$.

Estimation

There are two ways to estimate $\tau$ based on such idea.

The first approach is based on regression estimators $\hat{\mu}_1$ and $\hat{\mu}_0$ of $$\begin{align*} \mu_1(x)&= \mathbb{E}[\mathbb{E}[Y^{obs}\mid X=x,W=1]]\\ \mu_0(x)&=\mathbb{E}[\mathbb{E}[Y^{obs}\mid X=x,W=0]]. \end{align*}$$ Then we can compute the plug-in estimator: $$\hat{\tau}= \frac{1}{n}\sum\limits_{i=1}^n[\hat{\mu}_1(X_i)-\hat{\mu}_0(X_i)].$$ One approximately correct way to think about this is that we are using regression to impute the missing potential outcomes for each individual.

The second approach is based on Horvitz-Thompson estimator or the inverse propensity score estimator. Firstly, we define the propensity score by $$\pi(x)=\mathbb{P}(W=1\mid X=x),$$ which represents the probability that a unit with covariates x receives treatment. Note that, $$\begin{align*} \mathbb{E}[W\mid X=x]&= \pi(x)\\ \mathbb{E}[1-W\mid X=x]&=1-\pi(x). \end{align*}$$ Then we have $$\begin{align*} \tau &= \mathbb{E}[\mathbb{E}[Y(1)-Y(0)\mid X]]\\ &= \mathbb{E}\left[\mathbb{E}\left[\frac{Y(1)W}{\pi(X)}-\frac{Y(0)(1-W)}{1-\pi(X)}\mid X\right]\right]\\ &= \mathbb{E}\left[\mathbb{E}\left[\frac{Y^{obs}W}{\pi(X)}\mid X\right]\right] - \mathbb{E}\left[\mathbb{E}\left[\frac{Y^{obs}(1-W)}{1-\pi(X)}\mid X\right]\right], \end{align*}$$ which can be estimated as $$\begin{align*} \hat{\tau} &= \frac{1}{n}\sum\limits_{i=1}^n \left[\frac{Y_i^{obs}W_i}{\pi(X_i)}-\frac{Y_i^{obs}(1-W_i)}{1-\pi(X_i)}\right]. \end{align*}$$ In practice, we need to estimate $\pi(x)$ by regression.

Comments

We use potential outcomes to describe causality and causal inference coming from the work of Neyman (and later Rubin). However, there are also other languages. For example, (causal) directed graphs pioneered by Judea Pearl. While they focus more on a much harder problem of causal discovery, which requires even stronger assumptions.

Nonparametric Regression

2021-12-07T00:00:00-08:00

This post summarizes nonparametric regression.

Non-parametric Regression

Assume that we have data $\{(X_1,Y_1),\ldots,(X_n,Y_n)\}$ and our goal is to estimate the regression function $$r(x)=\mathbb{E}[Y\mid X=x].$$ Recall that we can estimate the cdf without distributional assumption by Glivenko-Cantelli Theorem; however, we need smoothness assumption on $r(x)$ for non-parametric regression.

For any estimator $\hat{r}$, we define the integrated squared loss by $$L(\hat{r},r)=\int_{\mathcal{X}}[\hat{r}(x)-r(x)]^2\mathrm{d}x,$$ which depends on both $(X_i,y_i)$'s. The risk is thus given by $$R(\hat{r},r)=\mathbb{E}[L(\hat{r},r)]=\int_{\mathcal{X}}b^2(x)\mathrm{d}x+\int_{\mathcal{X}}v(x)\mathrm{d}x,$$ by switching the integral and expectation, where $$\begin{align*} b(x) &= \mathbb{E}[\hat{r}(x)] - r(x)\\ v(x) &= \mathbb{E}[(\hat{r}(x) - r(x))^2]. \end{align*}$$

Suppose we knew the joint distribution over $(X,Y)$. Then an alternative measurement of risk can be the prediction error $$R(\hat{r})=\mathbb{E}[(Y-\hat{r}(X))^2],$$ which is minimized by $r(x)=\mathbb{E}[Y\mid X=x].$

Kernel Smoothing

We use the first risk measurement for the following analysis. We assume that there is only one covariate, while it can be readily generalized to the multivariate case. The estimator takes the form $$\hat{r}(x)=\sum\limits_{i=1}^nw_i(x)Y_i,$$ where the weights are defined by$$w_i(x)=\frac{K\left(\frac{x-X_i}{h}\right)}{\sum\limits_{j=1}^nK\left(\frac{x-X_j}{h}\right)},$$ $h$ is the bandwidth controlling the amount of smoothing, and $K$ is the kernel.

Jin-Hong Du

Academic Job Interview

Screening interview questions

Personal

Interest

Contribution

Contribution to department/school/university

Contribution to the field

Contribution other than research and teaching

Research

Teaching

DEI

Grant

My questions

Others

One-on-one interview questions

Communication and presentation

Resources

Python Handbook

Conda

Setup of R environment

Jupyter Notebook

Python kernel

R kernel

Bash commands

Running scripts

Python dependencies

R dependencies

Linux

System monitoring

File management

User management

Docker

VPN / Proxy Checking and Node Selection on Linux Server

1. Check whether mihomo is running

2. Check mihomo config path

3. Test the proxy endpoint

4. Check whether the controller API works

5. List all proxy groups

6. Show available nodes in the Proxy group

7. Manually switch node

8. Useful helper script: switch-node

9. Enable proxy variables for Claude / Codex / CLI tools

11. Minimal daily workflow

SOCKS5-based VPN on Linux

LaTeX on Linux

Slurm

LLM

Divergence Measures

Divergence Measures

Total variation distance

Squared Hellinger distance

Kullback–Leibler divergence

Chi-square divergence

Properties

Comparison

Tenserization

Example

Norm Inequality

Norm Equivalence

Useful Inequalities

Miscellanies

Reproducing Kernel Hilbert Spaces

Elementary Spaces

Linear Operators

Continuity

Duality

Isometric isomorphism

RKHS

Reproducing kernel

Inner product between features

Positive definite function

Differential Theory

Characterization of Smoothness

Directional Derivatives

Subgradients

Essential Smoothness and Essential Strictly Convexity

Smoothness and Strictly Convexity

The Legendre Transformation

Duality

6. Show available nodes in the `Proxy` group

8. Useful helper script: `switch-node`