We introduce xVal, a new number encoding scheme for LLMs. Using xVal with a modified number inference method makes LLMs continuous function approximators. This makes them have a better inductive bias for data analysis in scientific domains.

May 30, 2024

20 min read

ML

Counting in Context

We introduce the Contextual Counting task, a new toy problem aimed at exploring interpretability of Transformer models in quantitative domains.

Oct 9, 2023

15 min read

ML

xVal Number Encoding

We introduce xVal, a new number encoding scheme for LLMs. Using xVal with a modified number inference method makes LLMs continuous function approximators. This makes them have a better inductive bias for data analysis in scientific domains.

May 5, 2018

10 min read

ML

Backdrop: Stochastic Backprop

Backdrop is a proposed method for increasing stochasticity when performing gradient descent on objectives that are not decomposable over smaller minibatches.

Sep 22, 2017

5 min read

Physics

Particle-Hole Symmetry and Quantum Hall

We construct an effective field theory of composite Dirac fermions for the Jain sequences of quantum Hall plateaux. Using a Fermi liquid description, we derive the Girvin-Macdonald-Platzman algebra, calculate the dispersion of neutral excitations, and compute the electromagnetic response functions.

Dec 14, 2015

10 min read

Physics

Gravitational Anomalies in Field Theory

We analyze the effect of a global gravitational anomaly (that of the Dehn twist) and show that it has measurable consequences for low energy physics.

Feb 26, 2015

5 min read

Physics

Superfluids and Holography

Using a holographic setup we computed the hall viscosity of chiral p-wave superfluids and found that the zero temperature relationship between the superfluid density and Hall viscosity holds at all temperatures.

Jan 14, 2015

10 min read

Physics

Hall Viscosity, Spin Density, and Torsion

We clarify the relationship between Hall viscosity and space-time torsion by showing the consistent definition of these quantities in the first-order formulation of general relativity.

Sep 9, 2014

10 min read

Physics

Non-Relativistic CFT Operator Algebras

By analyzing the relationship between various operators in non-relativist conformal field theories, we derive a family of conservation laws.

Jul 28, 2014

5 min read

Physics

The Euler Current

We construct a relativistic current in odd dimensions whose conserved charge is the Euler characteristic in the codimension 1 even dimensional space.

Sep 10, 2013

10 min read

Physics

Magneto-Roton and Quantum Hall

Using a generic parametrization for the spectral densities of the energy-momentum tensor we find relationships between various physical properties of the system. These spectral densities can be described as the probabilities for absorption or emission of circularly polarized gravitons by the quantum Hall system.

Jul 24, 2012

5 min read

Physics

Chiral Vortical Effect Renormalization

We show that the coefficient of the Chiral Vortical Effect is fixed by quantum field theory loop calculation arguments.