Git Product home page Git Product logo

acl-org / acl-anthology Goto Github PK

View Code? Open in Web Editor NEW
379.0 21.0 253.0 310.92 MB

Data and software for building the ACL Anthology.

Home Page: https://aclanthology.org

License: Apache License 2.0

Shell 1.49% HTML 7.92% Perl 0.47% Python 87.40% Makefile 1.45% TeX 0.15% SCSS 0.80% Just 0.32%
acl acl-anthology computational-linguistics natural-language-processing library library-management-system

acl-anthology's Introduction

ACL Anthology

ACL Anthology Website GitHub contributors Good first project issues License acl-anthology-py on PyPI

This repository contains:

The official home of this repository is https://github.com/acl-org/acl-anthology.

Using the acl-anthology-py Python package

Please see the separate README for the Python package for detailed information.

Generating the Anthology website

These are basic instructions on generating the ACL Anthology website as seen on https://aclanthology.org/.

Prerequisites

To build the Anthology website, you will need:

  • Python 3.8 or higher
  • Python packages listed in bin/requirements.txt; to install, run pip -r bin/requirements.txt
  • Hugo 0.58.3 or higher (can be downloaded directly from their repo; the extended version is required!)
  • bibutils for creating non-BibTeX citation formats (not strictly required to build the website, but without them you need to invoke the build steps manually as laid out in the detailed README)
  • optional: If you install libyaml-dev and Cython before running make the first time, the libyaml C library will be used instead of a python implementation, speeding up the build.

Building and deployment with GitHub

There is a GitHub actions action performing deployment directly from GitHub. To use this, you need to define this variable in your repository settings (web interface: settings -> secrets):

  • PUBLISH_SSH_KEY: the secret key in standard pem format for authentication (without a passphrase)

GitHub will then automatically build and deploy the current master whenever the master branch changes. This is done via the upload target in the Makefile.

Cloning

Clone the Anthology repo to your local machine:

$ git clone https://github.com/acl-org/acl-anthology

Generating

Provided you have correctly installed all requirements, building the website should be as simple running make from the directory to which you cloned the repo.

The fully generated website will be in build/anthology afterwards. If any errors occur during this step, you can consult the detailed README for more information on the individual steps performed to build the site. You can see the resulting website by launching a local webserver with make serve, which will serve it at http://localhost:8000.

Note that building the website is quite a resource-intensive process; particularly the last step, invoking Hugo, uses about 18~GB of system memory. Building the anthology takes about 10 minutes on a laptop with an SSD.

(Note: This does not mean you need this amount of RAM in your system; in fact, the website builds fine on a laptop with 8 GB of RAM. The system might temporarily slow down due to swapping, however. The figure of approx. 18 GB is the maximum RAM usage reported when running hugo --minify --stepAnalysis.)

The anthology can be viewed locally by running hugo server in the hugo/ directory. Note that it rebuilds the site and therefore takes about a minute to start.

Hosting a mirror of the ACL anthology

First, creating a mirror is slow and stresses the ACL Anthology infrastructure because on initial setup you have to download every single file of the anthology from the official webserver. This can take up to 8 hours no matter how fast your connection is. So please don't play around with this just for fun.

If you want to host a mirror, you have to set two environment variables:

  • ANTHOLOGY_PREFIX the http prefix your mirror will be reachable under e.g. https://example.com/my-awesome-mirror or http://aclanthology.lst.uni-saarland.de (Notice that there is no slash at the end!)
  • ANTHOLOGYFILES the directory under which papers, attachments etc. will reside on your webserver. This directory needs to be readable by your webserver (obviously) but should not be a subdirectory of the anthology mirror directory.

With these variables set, you run make to create the pages and make mirror to mirror all additional files into the build/anthology-files directory. If you created a mirror before already, it will only download the missing files.

If you want to mirror the papers but not all attachments, you can run make mirror-no-attachments instead.

You then rsync the build/website/ directory to your webserver or, if you serve the mirror in a subdirectory FOO, you mirror build/website/FOO. The build/anthology-files directory needs to be rsync-ed to the ANTHOLOGYFILES directory of your webserver.

As you probably want to keep the mirror up to date, you can modify the shell script bin/acl-mirror-cronjob.sh to your needs.

You will need this software on the server

  • rsync
  • git
  • python3
  • hugo > 0.58
  • python3-venv

If you want the build process to be fast, install cython3 and libyaml-dev (see above).

Note that generating the anthology takes quite a bit of RAM, so make sure it is available on your machine.

Contributing

If you'd like to contribute to the ACL Anthology, please take a look at:

History

This repo was originally wing-nus/acl and has been transferred over to acl-org as of 5 June 2017.

License

The code for building the ACL Anthology is distributed under the Apache License, v2.0.

acl-anthology's People

Contributors

acl-pwc-bot avatar agutkin avatar akoehn avatar andreasvc avatar anthology-assist avatar aryamccarthy avatar chbrown avatar ctnlp avatar danielgildea avatar davidstap avatar davidweichiang avatar esalesky avatar guyaglionby avatar kilian-gebhardt avatar kite1988 avatar knmnyn avatar leondz avatar marcschulder avatar mbollmann avatar mcandito avatar mdelhoneux avatar mjpost avatar mlpacheco avatar nschneid avatar shashwatup9k avatar texttheater avatar villalbamartin avatar xinru1414 avatar yufei118liu avatar zamakkat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

acl-anthology's Issues

Number of items per page doesn't work if query contains number

Compare:

Queries for words without numbers in them work well (e.g. "framenet"), and I can select the number of items per page. But I cannot choose the number of items per page for queries like "flickr30k" without the website breaking down:

The page you were looking for doesn't exist.

You may have mistyped the address or the page may have moved.

If you are the application owner check the logs for more information.

CL titles not properly appearing in the Rails app.

Slightly related text:
I still see one problem with the CL articles:
in input files J10.xml through J16.xml, the journal title
"Computational Linguistics" is missing in the
header and therefore in the generated .bib files.

"Page doesn't exist" errors when searching by author name

I doubt this issue is not know but I couldn't find it in the issue list here on github:

When I search for an author by name, quite often I get "The page you were looking for doesn't exist." errors even though, there are papers by this author in the anthology. E.g. when looking for Hannah Rashkin (see Screenshot) I get this error, but there is a paper by her: http://aclanthology.info/papers/connotation-frames-a-data-driven-investigation.bib

acl anthology

rake:import_xml undesired behavior when using quotes

As seen in bug #51, giving a command-line argument for rake:import_xml that it's not exactly 3 characters long deletes and re-imports the entire database from scratch. This is technically correct, but wrong for all practical purposes.

  • Expected behavior: the command should give an error message, something like "that file does not exist, and therefore cannot be imported".
  • Actual behavior: the database is deleted and re-imported from scratch.

Paper pdf returning 403

The paper Assessing the Challenge of Fine-Grained Named Entity Recognition and Classification is returning a 403 when accessed through the Anthology interface. This happens because the paper's PDF has been moved, from http://www.aclweb.org/anthology/W10-2415.pdf to http://www.aclweb.org/anthology/W10-2415.old.pdf.

The paper can be still accessed through Google, but not through the Anthology. The person who pointed this out to me also mentioned that there's a revised version of it, available here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.357.7217&rep=rep1&type=pdf .

Should we replace the PDF link?

Character ń appears as ? in export files

The diacritic character ń is causing problems while generating export files. I have just checked my publications list:

http://aclanthology.info/catalog?utf8=%E2%9C%93&search_field=all_fields&q=Agnieszka+Falenska

The three papers which were published under surname "Faleńska" and not "Falenska" have errors in bibtex files (but also all other export files). There is "Fale?ska" without n on the authors list (it should be \'{n} ).

It might cause problems for anybody who would like to cite those papers. And also for any other authors who have ń in their surnames.

Search not looking into content of PDF files

The current search does not explore the text of the papers themselves, but only their metadata. As it has been pointed out in issue #39, this is less than ideal.

The search functionality would be greatly enhanced if we could look into the content of the PDF files themselves. This is likely to be quite complicated.

I'm opening this issue as a feature proposal, in order to collect ideas.

Add users to trial VM

We Now have a trial virtual machine with the computer linguistics group in Saarbrücken. All the volunteers should get accounts for this machine.

ERROR: cannot truncate a table referenced in a foreign key constraint

Did something change with respect to ingestion?

I didn't have this problem with re-ingesting proceedings but now it is happening. This is for W17.xml to change one author in W17-74.

aclanthology@aclanthology:~/acl-anthology$ rake import:xml["W17"] --trace
** Invoke import:xml (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute import:xml
PG::FeatureNotSupported: ERROR: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "papers_people" references "people".
HINT: Truncate table "papers_people" at the same time, or use TRUNCATE ... CASCADE.
: TRUNCATE TABLE people RESTART IDENTITY;
rake aborted!
ActiveRecord::StatementInvalid: PG::FeatureNotSupported: ERROR: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "papers_people" references "people".
HINT: Truncate table "papers_people" at the same time, or use TRUNCATE ... CASCADE.

ACL 2017 not indexing correctly on Google scholar.

The dates and the publisher information is not being picked up by google scholar. Also, a lot of the references are not indexed with the correct metadata (i.e., they don't get added to the list of citations of the papers they cite).

XML for J series missing title of journal

I still see one problem with the CL articles:
in input files J10.xml through J16.xml, the journal title
"Computational Linguistics" is missing in the
header and therefore in the generated .bib files.

I guess this is a problem with the web scraper?

Broken pdf links and incorrect url in bib files

Create a plan for mirrors.

In the not too distant future we should establish a number of mirrors that ensure that the core information of the anthology is always available. We will have to discuss exactly what these mirrors should do and where to get the resources for them.

P17-1.bib

Hi Min-Yen,

I think you are the maintainer of the ACL Anthology? I found an error in the ACL 2017 bibtex file which makes it unparseable. There is missing comma at the end of line 4491 of this file:

    http://aclweb.org/anthology/P/P17/P17-1.bib

Sincerely,

Author order incorrect just on paper list page (D17 specifically)

via email.

The author order seems likely to be a UI issue in the new anthology.
When I click on each of the papers, I see a different (correct) order of authors on the paper details page.
I also compared this to the old anthology. The old anthology appears to have the authors in the correct order too.

My guess is that the metadata has the correct order, but the web rendering is somehow reordering the authors on the page (only in paper list in the new anthology).

See screenshots:

DOI update

S/N: 3
Title: DOI update
Notes: synced at 16 Dec 2015

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.