If my filename contains one or more spaces (or, I suppose, any character the shell considers special), the program throws an error:

In [19]: fieldlist = pypdftk.dump_data_fields("test file.pdf")
Error: Unable to find file.
Error: Failed to open PDF file:
   test
Error: Unable to find file.
Error: Failed to open PDF file:
   file.pdf
Done.  Input errors, so no output created.
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-19-7f19f024450d> in <module>()
----> 1 fieldlist = p.dump_data_fields("test file.pdf")

/usr/local/lib/python3.5/dist-packages/pypdftk.py in dump_data_fields(pdf_path)
     94     # Or return bytes with : (will break tests)
     95     #    field_data = map(lambda x: x.split(b': ', 1), run_command(cmd, True))
---> 96     field_data = map(lambda x: x.decode("utf-8").split(': ', 1), run_command(cmd, True))
     97     fields = [list(group) for k, group in itertools.groupby(field_data, lambda x: len(x) == 1) if not k]
     98     return [dict(f) for f in fields]

/usr/local/lib/python3.5/dist-packages/pypdftk.py in run_command(command, shell)
     41 def run_command(command, shell=False):
     42     ''' run a system command and yield output '''
---> 43     p = check_output(command, shell=shell)
     44     return p.split(b'\n')
     45

/usr/local/lib/python3.5/dist-packages/pypdftk.py in check_output(*popenargs, **kwargs)
     35         if cmd is None:
     36             cmd = popenargs[0]
---> 37         raise subprocess.CalledProcessError(retcode, cmd, output=output)
     38     return output
     39

CalledProcessError: Command '/usr/bin/pdftk test file.pdf dump_data_fields' returned non-zero exit status 1

From the last line there I could see what the problem was: the filename is plopped into the command line unquoted. So I can work around the problem by including quote characters in the argument:

In [19]: fieldlist = pypdftk.dump_data_fields("'test file.pdf'")

I think this really should not be necessary. This behaviour would be confusing to new users, and makes the library more difficult to use.

After skimming through the source code, it looks like a minimal solution could be to simply add quote characters around the user-supplied filename arguments everywhere they are used. Quite likely there are other sorts of arguments that should be quoted on the command line also.

But I have another suggestion: why not just quote everything that goes on the command line? The run_command() function could take a list of items or tokens, and simply join them with a space between after adding the quote characters. Furthermore, it appears to me that the run_command() function is called with the same first argument every time. If this is true, why not include it in the function instead of requiring the caller to pass it? Obviously this would involve a lot more modification to the code, but I think it would simplify it on the whole, and potentially eliminate a whole class of possible bugs/user errors. I guess that would be a breaking change though for existing code that already adds quotes.

Hi! The concat function is breaking the input archive filename into single letters. I was unable to fix it.

Thanks.

python3 fillpdf.py ]Error: Unable to find file. Error: Failed to open PDF file: , Error: Unable to find file. Error: Failed to open PDF file: a Error: Unable to find file. Error: Failed to open PDF file: r Error: Unable to find file. Error: Failed to open PDF file: o Error: Unable to find file. Error: Failed to open PDF file: t Error: Unable to find file. Error: Failed to open PDF file: o Errors encountered. No output created. Done. Input errors, so no output created. Traceback (most recent call last): File "fillpdf.py", line 10, in <module> out_pdf = pypdftk.concat('c,garoto', 'mesclado.pdf') File "/home/lucas/.local/lib/python3.7/site-packages/pypdftk.py", line 116, in concat run_command(args) File "/home/lucas/.local/lib/python3.7/site-packages/pypdftk.py", line 43, in run_command p = check_output(command, shell=shell) File "/home/lucas/.local/lib/python3.7/site-packages/pypdftk.py", line 37, in check_output raise subprocess.CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command '['/usr/bin/pdftk', 'c', ',', 'g', 'a', 'r', 'o', 't', 'o', 'cat', 'output', 'mesclado.pdf']' returned non-zero exit status 1.

If it does not exist, give a warning or fail with a helpful message.

I won't go into the long story behind finding this issue...suffice it to say that a very strange bug silently appeared when running my app on Windows, caused by bad output from dump_data_fields(). At the root of the issue is line 44 of pypdftk.py, function run_command(): return p.split(b'\n')

This works on Linux/Unix because the newline character delineates lines of text. But on Windows you end up with \r at the end of each line. (Maybe on Mac it doesn't split anything at all, I don't know.)

This is not what the user expects, and depending what you use that output for it appears corrupted or causes other fun problems.

The quick-and-dirty fix would be to add .strip(b'\r') (or maybe .rstrip(b'\r')?) to the end of that line. Probably a better solution would be to encode the whole thing to a UTF-8 string then use .splitlines() on it. That should work across platforms, and has the advantage that returning unicode strings is probably better in most cases anyway...although it would affect existing code using this library. Also several other changes to this library would be required.

Thanks for your work on this and for making it available!

Hi
Getting an error like this, I did set the environment variables, but its still showing an error.

how can set the environment variables. please give me a detailed answer.

An image-to-stamp process would be useful. Stamp should have an invisible background and a reasonable default size.

Is it possible to fill forms with non-standard adobe fonts? ex. "Interstate" font if the form field has been defined in that font.
It seems that some fonts are not available to fill and are replaced with more standard fonts (like Arial).

What are the possible list of fonts available to use with fill_form?

I tried to write a function to create pdf_chunk files via pdftk syntax:

pdftk in.pdf cat 1-end output out.pdf

using:

def gen_pdfchunks(pdf_path, chunksize=5, first=1, last='end'):
    '''
    Generate chunks of large pdf files.
    '''
    cleanOnFail = True
    out_dir = tempfile.mkdtemp()
    chunk_file = '%s/chunk.pdf' % out_dir

    page = first
    if last == 'end':
        last_page = pypdftk.get_num_pages(pdf_path)
    else:
        last_page = int(last)

    # pdftk: pdftk in.pdf cat 1-5 output out1.pdf
    while page <= last_page:
        args = [PDFTK_PATH, pdf_path, 'cat']
        if page + chunksize - 1 > last_page:
            args += str(page)+'-'+str(last_page)
        else:
            args += str(page)+'-'+str(chunksize-1)
        args += ['output', chunk_file]
        try:
            run_command(args)
        except:
            if cleanOnFail:
                shutil.rmtree(out_dir)
            raise
        page += chunksize
        yield chunk_file

Calling

chunk_gen = gen_pdfchunks(file)
next(chunk_gen)

with a valid pdf file in varibale file the folling output is produced:

Error: Unexpected range end; expected a page
   number or legal keyword, here: 
   Exiting.
Errors encountered.  No output created.
Done.  Input errors, so no output created.
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-71-8f19f4570272> in <module>
----> 1 next(chunk_gen)

<ipython-input-69-9da5f665d933> in gen_pdfchunks(pdf_path, chunksize, first, last)
     22         args += ['output', chunk_file]
     23         try:
---> 24             run_command(args)
     25         except:
     26             if cleanOnFail:

<ipython-input-38-64212d8798aa> in run_command(command, shell)
     14 def run_command(command, shell=False):
     15     ''' run a system command and yield output '''
---> 16     p = check_output(command, shell=shell)
     17     return p.split(b'\n')
     18 

<ipython-input-38-64212d8798aa> in check_output(*popenargs, **kwargs)
      9         if cmd is None:
     10             cmd = popenargs[0]
---> 11         raise subprocess.CalledProcessError(retcode, cmd, output=output)
     12     return output
     13 

CalledProcessError: Command '['/usr/bin/pdftk', '/home/pfeifer/Multimedia/Bücher@LP/Akupunktur/Praxis Akupressur - Christina Mildt.pdf', 'cat', '1', '-', '4', 'output', '/tmp/tmp32l4j3vt/chunk.pdf']' returned non-zero exit status 1.

The problem is, that the given page range for the chunks does not keep one string, e.g. with chunksize=5 this should be 1-4 instead of 1 - 4 which is what the command in the CalledProcessError line is doing.

I just started to write in Python, but like to contribute to the project. I am not 100% confident that the code would do what I want, if the error not have been raised. In order to improve the project and my skills I like to ask for suggestions to clean/improve/correct my given code and of course help in developing the desired function.

Best regards,
Sebastian

#12 is a py3 issue.

Traceback (most recent call last):
  File "C:/Users/Rhea/Documents/GitHub/LeaveFormFiller/leaveForm.py", line 395, in submit_leave
    self.leave_form.create_form()
  File "C:/Users/Rhea/Documents/GitHub/LeaveFormFiller/leaveForm.py", line 161, in create_form
    fill_form("leaveForm.pdf", self.datas )
  File "C:\Python34\lib\site-packages\pypdftk.py", line 55, in fill_form
    tmp_fdf = gen_xfdf(datas)
  File "C:\Python34\lib\site-packages\pypdftk.py", line 129, in gen_xfdf
    f.write(tpl.encode('UTF-8'))
TypeError: must be str, not bytes

Best I can tell, everything I am sending to fill_form is a 'str'. I went through the debugger just to make sure, there isn't anything that isn't a str on my end. It still may be my fault. I was using fdfgen and it uses a list instead of a dict. So i had to change all of that.

Edit: This using python 3.4 not sure if that is the issue or not.

While debugging issue #37 , I noticed that dump_data_fields does not support multi-line FieldValues atm. This is due to the logic in run_command where the result of check_output is split into individual lines. Given that text fields can contain newline/carriage return characters, this logic should be adapted.

The results of dump_data_fields() does not give me all the information I need from the PDF file. For instance (using the US IRS Form 941 as an example), here is my code and a section of the output:

In [27]: fieldlist = pypdftk.dump_data_fields("f941-2019.pdf")

In [28]: fieldlist[18:22]
Out[28]:
[{'FieldFlags': '0',
  'FieldJustification': 'Left',
  'FieldName': 'topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[0]',
  'FieldStateOption': 'Off',
  'FieldType': 'Button',
  'FieldValue': 'Off'},
 {'FieldFlags': '0',
  'FieldJustification': 'Left',
  'FieldName': 'topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[1]',
  'FieldStateOption': 'Off',
  'FieldType': 'Button',
  'FieldValue': 'Off'},
 {'FieldFlags': '0',
  'FieldJustification': 'Left',
  'FieldName': 'topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[2]',
  'FieldStateOption': 'Off',
  'FieldType': 'Button',
  'FieldValue': 'Off'},
 {'FieldFlags': '0',
  'FieldJustification': 'Left',
  'FieldName': 'topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[3]',
  'FieldStateOption': 'Off',
  'FieldType': 'Button',
  'FieldValue': 'Off'}]

But if I run PDFtk from the shell, it shows another "FieldStateOption" for each of these checkboxes. Here's the corresponding output of pdftk f941-2019.pdf dump_data_fields:

---
FieldType: Button
FieldName: topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[0]
FieldFlags: 0
FieldValue: Off
FieldJustification: Left
FieldStateOption: 1
FieldStateOption: Off
---
FieldType: Button
FieldName: topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[1]
FieldFlags: 0
FieldValue: Off
FieldJustification: Left
FieldStateOption: 2
FieldStateOption: Off
---
FieldType: Button
FieldName: topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[2]
FieldFlags: 0
FieldValue: Off
FieldJustification: Left
FieldStateOption: 3
FieldStateOption: Off
---
FieldType: Button
FieldName: topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[3]
FieldFlags: 0
FieldValue: Off
FieldJustification: Left
FieldStateOption: 4
FieldStateOption: Off

So I understand that the reason this happens is because pypdftk is putting the output of PDFtk into a list of dictionaries, so naturally the later values for a given key overwrite the earlier values. But the fact is that data is lost, and in this case it is precisely the data I need. (The "FieldStateOption" that isn't "Off" is the one I have to use to "check" the checkbox. Note that it is different for each field, which is why I want my program to read it. In this case it comes first; apparently it doesn't always. See this StackExchange discussion.)

My suggestion would be doing a little more sophisticated processing of the PDFtk output, so that if a key is repeated, its value in the resulting dictionary would be a list. Then the result in Python would look like this in my case — taking the first item in the example above:

[{'FieldFlags': '0',
  'FieldJustification': 'Left',
  'FieldName': 'topmostSubform[0].Page1[0].Header[0].ReportForQuarter[0].c1_1[0]',
  'FieldStateOption': ['1', 'Off'],
  'FieldType': 'Button',
  'FieldValue': 'Off'}]

As of this writing, the section pdftk path of README.md provides the following.

By default, path is /usr/bin/pdftk, but you can override it with the PDFTK_PATH environment variable

I would like to suggest the following revision. It's longwinded, I know; but worth it (I think/hope).

This module, being a driver, requires that you already have pdftk installed. First, the module will look to see if the PDFTK_PATH environment variable is set. If it's not set, it'll look in the /usr/bin directory. As last resort, it'll simply look in whatever directory it, i.e., the module itself, is located.
Also: Double-check your PDFTK_PATH environment variable, and make sure it ends with the filename of the binary itself, e.g., pdftk and not just the path to the directory where it is located.
For example (on a Windows machine, where pdftk is installed in C:\Program Files (x86))...
If PDFTK_PATH is set to C:\Program Files (x86)\PDFtk\bin, it will throw the test call error; but, if you add pdftk, i.e., binary's filename, to the end, it will not.
e.g., C:\Program Files (x86)\PDFtk\bin\pdftk

Or... you could just add a check for that in the module itself. Maybe something like...

if os.getenv('PDFTK_PATH'):
    PDFTK_PATH = os.getenv('PDFTK_PATH')
    if not PDFTK_PATH.endswith('pdftk'):
        PDFTK_PATH = os.path.join(PDFTK_PATH, 'pdftk')
else:
    PDFTK_PATH = '/usr/bin/pdftk'
    if not os.path.isfile(PDFTK_PATH):
        PDFTK_PATH = 'pdftk'

I want to create an API that just serves a PDF as Base64 for a browser app. Do i always have to save the file and then process it or can this library just return the data? Would like to not store any customer data if possible.

I am looking for a way to embed images in the fillable PDF. Is there any possibility with this?

Working with your library extensively today (it's great!), I noticed that whenever the pdftk subprocess crashes, some artifacts are not being cleaned up from /tmp.

Here is a proof that asserts that in case of a pdftk crash, the generated FDF remains in /tmp:

import os
import re
import subprocess

from pypdftk import fill_form

datas = {
    # this is gonna crash pdftk, because the angle bracket is not escaped
    'Given Name Text Box': '1 < 2'
}

try:
    fill_form('./OoPdfFormExample.pdf', datas=datas, out_file='out.pdf')
except subprocess.CalledProcessError as e:
    fdf_file = re.match(r'^.+(/tmp/.+?)\s', str(e)).group(1)

    assert os.path.exists(fdf_file)

(The PDF sample is taken from this source)

An opening angle bracket makes pdftk crash, because it's rendered into the XML as-is and not escaped (that's an issue of it's own - gonna file that too).

The fill_form function looks like this:

def fill_form(pdf_path, datas={}, out_file=None, flatten=True):
    cleanOnFail = False
    tmp_fdf = gen_xfdf(datas)
    handle = None
    if not out_file:
        cleanOnFail = True
        handle, out_file = tempfile.mkstemp()
    ...
    try:
        run_command(cmd, True)
    except:
        if cleanOnFail:
            os.remove(tmp_fdf)
        raise
    finally:
        if handle:
            os.close(handle)
    os.remove(tmp_fdf)
    return out_file

If run_command(cmd, True) raises, first the except, then the finally block are executed. Because the except block re-raises, the last two lines (after the finally) block are never executed. So in case cleanOnFail is False (when outfile is being passed as an argument), the tmp_fdf is never removed.

There is another bug hiding in this: If outfile is not being passed, a temporary file is created and a handle is opened:

 if not out_file:
        cleanOnFail = True
        handle, out_file = tempfile.mkstemp()

In the finally block, the handle is closed, but the out_file is not being removed (although the failing pdftk already created the file, even though it's 0 bytes. It almost looks like a copy-paste problem, that if cleanOnFail is True, actually os.remove(out_file) should have been called?

I can fix both problems and create a PR, if you like.

In the example there is pypdftk.merge example for merging two pdf. When trying it I receive an error:

AttributeError: 'module' object has no attribute 'merge'

When I look it source code I also don't see merge function.

Could you please provide more examples on using the features of this tool. It sounds like it is exactly what I need but I am not exactly sure how to write it out.
I am looking to fill in a few forms, flatten only the ones I fill in and overwrite the file I opened once filled in.

I am also getting the error 'must be str, not bytes' and am looking for how to convert?

strung = 'example.pdf'
thispath = os.path.abspath(strung)
print(thispath)
mypath = pypdftk.get_num_pages(thispath)
print(mypath)

Traceback (most recent call last):
File "C:\freeopcua\examples\40T.py", line 41, in
mypath = pypdftk.get_num_pages(thispath)
File "C:\Python34\lib\site-packages\pypdftk.py", line 43, in get_num_pages
for line in run_command([PDFTK_PATH, pdf_path, 'dump_data']):
File "C:\Python34\lib\site-packages\pypdftk.py", line 37, in run_command
p = check_output(command, shell=shell)
File "C:\Python34\lib\site-packages\pypdftk.py", line 24, in check_output
process = subprocess.Popen(stdout=subprocess.PIPE, _popenargs, *_kwargs)
File "C:\Python34\lib\subprocess.py", line 848, in init
restore_signals, start_new_session)
File "C:\Python34\lib\subprocess.py", line 1104, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

Hi, I need to run the script with flatten set to False, but also with commands "need_appearances, drop_xfa", or else it doesn't encote UTF-8 characters properly. How do I do that with pypdftk?

Hi,
is it possible to add the background option in your package as

https://stackabuse.com/working-with-pdfs-in-python-adding-images-and-watermarks/
Adding a Watermark via Command Line with pdftk

I had an AWS Lambda Function that ran on pypdftk 0.4, and Python 2.7
Both versions are deprecated and I need to update.

Hello, can you please release 0.4 version to pypi ?

Thanks!

Is it possible to change font type and size for the text that is going to be added to the pdf using this module? If so, how? Reference: https://stackoverflow.com/questions/62632939/using-a-different-font-with-pdftk

Hi,

I'm not an experienced coder, and I'm trying to use this package. I noted that there is an issue with python 3 that is being looked at.

However, I am also having an issue using the path. I noted that the program doesn't like it when I use the whole path starting with C:\ . It also doesn't like it when I just use the name of the file, which is in the same folder as the scratch file. I was hoping I could get some more direction in terms of what this should look like.

My version of pdftk (3.0.2) has both a dump_data_fields and a dump_data_fields_utf8.
The former does not output accent correctly while the later does the job as expected.

Could/should it be exposed ?
Should the dump_data_fields python function use silently the dump_data_fields_utf8 command when available since in Python3 all string are unicode anyway ?

I want to use your package but I didn't found it in PyPI.

Can we add hyperlinks in some keyword using Pypdftk?