Git Product home page Git Product logo

tablefill's Introduction

tablefill

Automatically update LaTeX, LyX, and Markdown tables and numbers using a placeholder system.

Quickstart

pip install git+https://github.com/mcaceresb/tablefill
tablefill --help

See the getting started section of the documentation for an example in LaTeX of how to use this system.

Documentation

tablefill is a program that uses a generic placeholder system to update LaTeX, LyX, and Markdown documents. While it's original use was to fill in tables (hence the name) placeholders are replaced anywhere in a table environment or anywhere within commented-out tablefill tags. Hence any text or numbers anywhere in the document can be updated using this system.

The name and idea for this workflow comes from GSLab. This version was written to update LaTeX files in the same fashion, and later expanded to include Markdown and several features.

  • Getting Started gives a basic example of how the system works, and guides the user through updating a template using Stata-generated input files (this example can be replicated with any programming language, however, not just Stata).

  • The usage section details the input required by tablefill, the format of the placeholders that are allowed, and documents the tablefill usage options.

  • Last, we provide examples of how to generate the type of input required by tablefill in the sample programs page.

Background

When I came across the original tablefill it could only update LyX files with Stata output (though I believe LaTeX support has been added in the time since). The idea was to expand the system to work with LaTeX, but over time I re-wrote the code base and added several features, including Markdown support. For compatibility with scripts using GSLab's tablefill, the script can be run from the command line or imported as a python module. The workflow is as follows:

  1. Create a LaTeX, LyX, or Markdown document using placeholders. Label or tag each table appropriately.

  2. Print a matrix (or any array) into a tab-delimited text file. The output is preceded by the label or tag of the table you want to fill, and the matrix or array entries correspond to the placeholders.

  3. Run tablefill to update the placeholders in the template document.

Installation

pip install git+https://github.com/mcaceresb/tablefill
tablefill --help

This was created specifically to run in a server that only had Python 2.6 available. The function should be compatible with Python 2.6, Python 2.7, and Python 3.X without requiring the installation of any additional packages (the --numpy-syntax option is available if the function can successfully run import numpy; however, numpy is not required for normal use).

Features

As mentioned above, the original idea for tablefill comes from GSLab. However, this version has enough distinct features to merit calling itself a separate program. These include:

  • Ability to fill LaTeX and Markdown templates

    • Placeholders can also be placed between commented-out tablefill tags, so updating is not restricted to literal tables.
    • There can be several placeholders in one line (however, there must be at most one table line per code line).
    • Labels in LaTeX can be anywhere in the table environment.
    • Placeholders can be either # or \# (note the former is a special character in LaTeX so while the filled output will compile, the template will not).
    • Can have several matches of the pattern in the same line.
    • LaTeX tables can be filled inside Markdown documents.
    • The user can choose whether or not to fill in placeholders in commented-out lines.
  • Several error and consistency checks

    • Checks inputs are correct (names and type)
    • Checks if input files exist
    • Soft warning when placeholder outside proper environment
    • Soft warning when placeholder in environment with no label
    • Soft warning when placeholder in environment with unmatched label
    • Soft warning when more placeholders than table entries
  • Several new placeholder types:

    • Arbitrary python formatting via #{.*}# (note that in python 2.6 this must be #{0:.*}#).
    • #*# interprets the input as a p-value and replaces it with significance symbols (default is * 0.1, **0.05, ***0.01; however, anything that LaTeX, LyX, or Markdown will compile can be passed).
    • Modifier % reads the input as percentage (multiplies by 100).
    • Modifier || outputs the absolute value of the input.
  • The input tables can be generated by any program as long as they write missing values as blank, a dot, NA, or NaN.

  • tablefill can be run from the command line directly or imported to a python script.

  • Basic compilation support via the --compile and/or --bibtex flags.

  • XML-based ad-hoc table combination engine.

    • The user can combine arbitrary entries of existing tables in the provided templates to create new tables. This is useful if generating new tables is costly. It is also useful to quote table statistics in a summary elsewhere in the text (the LaTeX engine can parse the placeholders as long as they are in a table environment, which can be regular text).

Contributors

  • Kyle Barron has made several additions and improvements to this projects.
  • The idea behind this system came from from GSLab's tablefill.

License

MIT

tablefill's People

Contributors

causalmind avatar mcaceresb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

tablefill's Issues

UnboundLocalError: local variable 'tag' referenced before assignment

Just trying to create a simple example for the new RA... Am I doing something horribly stupid?

Using master:

> tablefill -i matrix.txt -o filled.tex template.tex
ERROR!
Traceback (most recent call last):
  File "/home/kyle/local/anaconda3/lib/python3.6/site-packages/tablefill/tablefill.py", line 337, in tablefill
    fill_engine.get_parsed_tables()
  File "/home/kyle/local/anaconda3/lib/python3.6/site-packages/tablefill/tablefill.py", line 793, in get_parsed_tables
    ctables[tag] += [clean_row_entries]
UnboundLocalError: local variable 'tag' referenced before assignment

ERROR while filling table. Check function call.

usage: tablefill.py [-h] [-v] [-i [INPUT [INPUT ...]]] [-o OUTPUT]
                    [-t {auto,lyx,tex}] [--pvals [PVALS [PVALS ...]]]
                    [--stars [STARS [STARS ...]]] [-f] [-c] [-b] [-fc]
                    [--ignore-xml] [--legacy-parsing] [--numpy-syntax]
                    [--use-floats] [--xml-tables [INPUT [INPUT ...]]]
                    [--verbose] [--silent]
                    TEMPLATE

matrix.txt:

example_matrix
         1	         2	         3
         3	       2.4	    2234.4
      1.32	     2.456	  3.345345
         3	       2.4	    2234.4

template.tex:

\documentclass{article}
\usepackage{tabularx}
\usepackage{booktabs}
\begin{document}
\begin{table}
  \caption{Table caption}
  \label{tab:example_matrix}
  \begin{tabular}{
  p{4.25cm}@{\hskip 6pt}c
  @{\hskip 15pt}
  r@{\hskip 3pt}c
  @{\hskip 15pt}
  c}
    Outcomes
    & N
    & Mean
    & (Std.)
    \\
    Outcomes 1 & \#0,\# & \#2,\# & (\#2,\#) \\
    Outcomes 2 & \#0,\# & \#2,\# & (\#2,\#) \\
    Outcomes 3 & \#0,\# & \#2,\# & (\#2,\#) \\
    Outcomes 4 & \#0,\# & \#2,\# & (\#2,\#) \\
    \multicolumn{4}{p{5cm}}{\footnotesize Footnotes here!}
  \end{tabular}
\end{table}
\end{document}

Parsing very long files fails?

Just tried parsing a file 10s of thousands of lines and the program said the tags were missing. I copied the missing tags to a smaller file and that did the trick. Check up on this,

Python 3 support?

I'm not sure if it was designed to support both Python 2 and 3, but I get errors when trying to run it with Python 3.

Running python fill_tables.py, where fill_tables.py contains:

#! /usr/bin/env python2

import glob
import os
from tablefill import tablefill

filled_names = glob.glob('filled/*.tex')
for i in filled_names:
    os.remove(i)

# Fill all latex tables in template/ except master.tex
names = [os.path.basename(x) for x in glob.glob('template/*.tex')]
for name in names:
    try:
        tablefill(template   = os.path.join("template", name),
                  output     = os.path.join("filled", name),
                  fillc      = True,
                  input      = "out/tables/tables.txt",
                  xml_tables = "parsing.xml")
    except:
        raise
Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Parsing arguments...
ERROR!
Traceback (most recent call last):
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 321, in tablefill
    fill_engine.get_parsed_arguments(kwargs)
  File "/home/kyle/research/doyle/eosloan/geisinger/local/miniconda/lib/python3.6/site-packages/tablefill/tablefill.py", line 673, in get_parsed_arguments
    isare = " is " if len(missing_args) == 1 else " are "
TypeError: object of type 'filter' has no len()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.