Hierarchical Content Understanding

This work was done during a 3-month summer internship where I researched geometric spaces and their ability to store information for processing. I developed an algorithm for hierarchical topic modeling by combining information from embeddings in hyperbolic and Euclidean space. I also worked on automatically labeling the topics produced in the hierarchy. We were able to beat baselines in terms of topic interpretability, coherence, granularity across the hierarchy, topic diversity, and memory and time efficiency. I published a findings paper at ACL 2023 (Srikanth et al., 2023) on this, a poster (Shahid et al., 2021) at NeurIPS 2021 WiML workshop, and a patent (Srikanth et al., 2024). I also developed an algorithm to label these hierarchical topics. This research was used to understand the readability and depth of content with respect to various reader segments.

References

2024

patent

Hierarchical topic model with an interpretable topic hierarchy

Nikitha Srikanth, Simra Shahid, Tanay Anand, and 3 more authors

Apr 2024

HTML

2023

paper

HyHTM: Hyperbolic Geometry-based Hierarchical Topic Model

Nikitha Srikanth, Simra Shahid, Tanay Anand, and 3 more authors

In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

Abs DOI

Hierarchical Topic Models (HTMs) are useful for discovering topic hierarchies in a collection of documents. However, traditional HTMs often produce hierarchies where lower-level topics are unrelated and not specific enough to their higher-level topics. Additionally, these methods can be computationally expensive. We present HyHTM - a Hyperbolic geometry-based Hierarchical Topic Model - that addresses these limitations by incorporating hierarchical information from hyperbolic geometry to explicitly model hierarchies in topic models. Experimental results with four baselines show that HyHTM can better attend to parent-child relationships among topics. HyHTM produces coherent topic hierarchies that specialize in granularity from generic higher-level topics to specific lower-level topics. Further, our model is significantly faster and leaves a much smaller memory footprint than our best-performing baseline. We have made the source code for our algorithm publicly accessible.

2021

poster

Interpretable & Hierarchical Topic Models using Hyperbolic Geometry

Simra Shahid, Tanay Anand, Nikitha Srikanth, and 3 more authors

In NeurIPS 2021 WiML Workshop 1, Dec 2021

Abs HTML

Topic Models (TM) are statistical models to learn latent topics present in a collection of text documents. These topics are usually not independent and represent concepts that are related hierarchically. Flat TMs such as LDA fail to capture this inherent hierarchy. To overcome these limitations, Hierarchical Topic Model (HTM) have been proposed that discover latent topics while preserving the inherent hierarchical structure between different topics (for example, aspect hierarchies in reviews and research topic hierarchies in academic repositories). Despite showing great promise, the state-of-the-art HTMs fail to capture coherent hierarchical structures. Also the number of topics in each level is usually unknown and is determined empirically, wasting a lot of time and resources. Finally, HTMs have very long training time, making them unsuitable for real-time production environments. Thus, there is a need for HTMs that (i) offers good hierarchical structures and more meaningful and interpretable topics and that (ii) can automatically find the number of topics at each level without multiple training iterations. In our work, we address the problems mentioned above by utilizing properties of hyperbolic geometry that has been successfully applied in learning hierarchical structures such as ontologies in Knowledge bases, and latent hierarchy between words. Our initial experiments have yielded promising results where the training time has reduced from weeks to less than an hour, and the quantitative metrics have improved with significantly better hierarchical structures. We have attached an abstract with a brief overview of our approach and results.