This is an easy-to-follow intro to hyperbolic geometry.
If you know nothing about it, this is for you! 🙂
The only parts which may require basic differential geometry are these
What’s a geometry?
It is commonly defined by a
There are many
A set together with a distance function is called a metric space.
We consider the geometries defined by two metric spaces and to be equivalent if they are
The hyperbolic space is a particular case of metric space with many interesting properties that can be very powerful for data representations in machine learning.
What’s hyperbolic geometry?
In differential geometry, the spaces that people study are called manifolds, a sort of high-dimensional generalization of curved surfaces.
Each point of such a manifold can be assigned a curvature. When the curvature is constant, it is either everywhere positive, zero or negative. This gives rise to three types of geometry: elliptic, Euclidean and hyperbolic respectively, usually considered as the
There exist 5 models of hyperbolic space, i.e. ways to represent it and work with it. The most commonly used in machine learning are the Poincaré (disk) model and the Lorentz (hyperboloid) model.
Their sets and distance functions are defined in the table below.
Where is called the Lorentz inner-product, and is called the conformal factor. These are also useful to define the
These two models are equivalent, i.e. they are isometric. See Figure 6 below.
It is hence easy to switch from one model of hyperbolic space, to another.
Now that we have defined hyperbolic spaces, let’s try to understand them. For intuition, the Poincaré model is the friendliest one. The following gif illustrates how “straight-lines” look like in the Poincaré disk.
As can be seen, around the origin of the disk, these lines look straight, as in Euclidean space.
However, the closer they get to the border, the stronger they bend.
Note that these lines are geodesics, i.e. they are shortest paths between the two points that they connect, and their length in hyperbolic space gives the hyperbolic distance between the two points.
Let’s have a closer look at the
If are close to the origin, then
This shows that close to the origin, the hyperbolic space resembles Euclidean geometry.
The hyperbolic nature of the space is more prominent at the border of the disk, which corresponds to points at infinity in the Lorentz model, according to Figure 6.
Moreover, in the Lorentz model , it is known that
The volume of a ball grows exponentially with the radius!
In comparison, the volume of in grows only proportionally to .
This exponential volume growth of hyperbolic spaces is reminiscent of trees.
Indeed, a binary tree of depth contains nodes.
This dilatation of distances can also be understood intuitively from the negative curvature of the space.
Consider the surface in Figure 9. It is negatively curved.
This means that it is stretched in opposite directions.
Now consider two rays of light emerging from the origin, shooting in different directions, but remaining inside the surface.
Because of negative curvature, they will go apart much faster if their respective directions are orthogonal, than if they are co-linear.
As a byproduct, the shortest path between the end points of these two rays of light will almost go near the origin.
This is similar to what the metric would look like in a tree, in which joining two leaves requires passing through the closest parent.
Similarly, one can think of a hyperbolic space as stretching the metric in opposite directions, connecting together the concepts of exponential volume growth, continuous tree-likeness and negative curvature…
What’s better in hyperbolic geometry?
Because hyperbolic spaces possess this exponential volume growth property, they are better suited to embed tree-like graphs, or any kind of data with an underlying
But is hyperbolic geometry only useful for hierarchical data?
No! It turns out that its power applies to any kind of data with an
At this stage, you may wonder:
Could we use hyperbolic embeddings in downstream task, i.e. as inputs to neural networks?
Could we design neural networks with hidden states in hyperbolic space?
The answer in the next episode… 🙂