FORMULAS about unarxive HOT 1 CLOSED

Hi,
thank you for the input.
The data set is derived from the LaTeX source files, which are parsed using Tralics. Given that a large portion of arXiv is physics and mathematics documents (and therefore contains a lot of formulas), we did consider retaining formulas in some form. However, as the primary focus during development was on textual contexts of citation markers, we opted to replace formulas with a simple placeholder token for the moment. Keeping formulas in some markup format (e.g. MathML) or the origial LaTeX or placing links to these contents in the generated plain text files is an option we're considering for the future though.

from unarxive.