I would like to turn any PDF into full ASCII via applying this filter on all streams.

I saw that mutool has an clean -a option to do so, but it doesn't do it automatically on every stream, which is odd.

I can't find any reference of ASCIIHexDecode on PyMuPDF or its utilities.
Am I missing something?

{ "spans":[], "wmode":1, "dir":[ 0.0, 1.0 ], "bbox":[ 0.0, 0.0, 0.0, 0.0 ] },

Make sure the examples are PEP8 compliant during and after the restructuring process.

We are facing some errors during layout extraction. On some PDFs, we are getting RuntimeError: program error: minslot too small = 0. I have taken a fast look over compute_slots and didn't see why it could be generating a slot with width 0. I found it happen on this line. Last written text is 637.

Hey,

I was browsing through the simpleGui examples and I was wondering what you thought about this idea:
very often, I want to open multi-columns PDF article on an old e-reader, with a smaller resolution.
For that, it makes more sense if only one column was displayed.
The content is the same, but it's just that the pages have been changed to rely on /CropBox, and there are more of them.
It could be automated across pages, but sometimes, there is a horizontal picture that takes the whole width of the page. In this case, a dedicated GUI should be required to show it.

WDYT? Do you know anything like this? Any better solution?

I want to get the first page in a single statement, as in:

>>>import fitz
>>>from pathlib import Path
>>>
>>>infile = Path("./some_document.pdf")
>>>page = fitz.Document(infile).load_page(0)
# raises ValueError: orphaned object: parent is None

My configuration:
python version: 3.9.5, mypupdf installed with pip

fitz.__doc__
PyMuPDF 1.18.16: Python bindings for the MuPDF 1.18.0 library.
Version date: 2021-08-05 00:00:01.
Built for Python 3.9 on linux (64-bit).

This script: https://github.com/pymupdf/PyMuPDF-Utilities/blob/master/examples/export-toc/export.py.

Any opinion what to do in this case? (maybe just use proper csv.writer will fix the issue?)

The byte appears as a ^M (CR character) in the generated text file.

(I suspect similar issue happen when there's quote character etc. in the bookmark title as well)

Hello,
I would like to extract the supplier's information (top left block: Bourgeois Frères, . (see attached image) from the attached pdf.
What is the solution?
Os : MacOs Bigsur 11.6
Python : 3.9
PyMuPDF : 1.19.0

Thank you.

Sincerely

BOURGEOISFacture 21053886.pdf

y

Some documents such as this one cause fitzcli.py to error.

#  python3 fitzcli.py gettext G*\).pdf
Traceback (most recent call last):
  File "fitzcli.py", line 1206, in <module>
    main()
  File "fitzcli.py", line 1202, in main
    args.func(args)  # execute requested command
  File "fitzcli.py", line 901, in gettext
    flags=flags,
  File "fitzcli.py", line 866, in page_layout
    textout.write((text + "\n").encode("utf8"))
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 39-42: surrogates not allowed

See #88

A markdown table containing a brief description of the updated examples should be added to the examples/README.md file.

Describe the bug (mandatory)

It looks like the interface to _getCharWidths() has changed.
PDF conversion results for file 'test.file': 1 lines read, 1 lines written, 61 lines per page. test.file.pdf contains 1 pages. Traceback (most recent call last): File "PyMuPDF-Utilities-master\PyMuPDF-Utilities-master\demo\text2pdf.py", line 73, in <module> fwidths = doc._getCharWidths("Helvetica") # get glyph widths of font TypeError: _getCharWidths() missing 4 required positional arguments: 'bfname', 'ext', 'ordering', and 'limit'

To Reproduce (mandatory)

echo "Test string" > test.txt
python PyMuPDF-Utilitites/demo/text2pdf test.txt

Expected behavior (optional)

Creation of PDF from input file.

Screenshots (optional)

If applicable, add screenshots to help explain your problem.

Your configuration (mandatory)

• Windows 10 64 bit
• Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
• pymupdf-1.16.10 -- PyMuPDF-1.16.10-cp38-none-win_amd64.whl
• PyMuPDF-Utilities commit: 7604352

Additional context (optional)

Hi,Ineed some some help,

When I try import images form PDF file

I find some image become some color loss,

origin target page looks like:

this is my test pdf file:
9骆驼618.pdf

page9 , page 16 some of imge is right,

i use pymupdf utilities

def recoverpix(doc, item):
    xref = item[0]  # xref of PDF image
    smask = item[1]  # xref of its /SMask

    # special case: /SMask or /Mask exists
    if smask > 0:
        pix0 = fitz.Pixmap(doc.extract_image(xref)["image"])
        mask = fitz.Pixmap(doc.extract_image(smask)["image"])
        pix = fitz.Pixmap(pix0, mask)
        if pix0.n > 3:
            pix = fitz.Pixmap(fitz.csRGB, pix)

        return {  # create dictionary expected by caller
            "ext": "png",
            "colorspace": pix.colorspace.n,
            "image": pix.tobytes(),
        }

    # special case: /ColorSpace definition exists
    # to be sure, we convert these cases to RGB PNG images
    if "/ColorSpace" in doc.xref_object(xref, compressed=True):
        pix = fitz.Pixmap(doc, xref)
        pix = fitz.Pixmap(fitz.csRGB, pix)
        return {  # create dictionary expected by caller
            "ext": "png",
            "colorspace": 3,
            "image": pix.tobytes("png"),
        }
    return doc.extract_image(xref)

pageitemindex = 0
for pageitem in page_data_objects:
    pageblocks = pageitem.get_text("dict")["blocks"]
    pagepath = os.path.join(debug_out_put_folder, page_gen_data["genpages"][pageitemindex]["folder"])
    print("gen layout :" + pagepath)
    pageoutputfilename = "layout.json"
    pageitemindex = pageitemindex + 1

    xreflist = []
    dimlimit = 0  # 100  # each image side must be greater than this
    relsize = 0  # 0.05  # image : image size ratio must be larger than this (5%)
    abssize = 0  # 2048  # absolute image size limit 2 KB: ignore if smaller

    for imgnumber, img in enumerate(pageitem.getImageList(), start=1):
        xref = img[0]

        xref = img[0]
        if xref in xreflist:
            continue
        width = img[2]
        height = img[3]
        if min(width, height) <= dimlimit:
            continue
        image = recoverpix(document, img)
        n = image["colorspace"]
        imgdata = image["image"]

        if len(imgdata) <= abssize:
            continue
        if len(imgdata) / (width * height * n) <= relsize:
            continue

        imgfile = os.path.join(pagepath, "img%05i.%s" % (xref, image["ext"]))
        fout = open(imgfile, "wb")
        fout.write(imgdata)
        fout.close()
        xreflist.append(xref)

thanks.

First, thanks for implementing such a great package. It is very helpful.

We are trying to run the tesseract2.py example code on the attached file and not getting any output. From inspecting the variables, it seems that the line "blocks = page.get_text("dict", flags=0)["blocks"]" is not returning any object, suggesting to us that the problem lies in the interaction with tesseract. We note that we are able to convert manually the attached file to txt using pdf2image and tesseract in the same environment.

Our configuration is as follows:

$ echo $TESSDATA_PREFIX
/usr/share/tesseract-ocr/4.00/tessdata
Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import fitz
print(fitz.doc)

PyMuPDF 1.19.4: Python bindings for the MuPDF 1.19.0 library.
Version date: 2022-01-01 00:00:01.
Built for Python 3.10 on linux (64-bit).
microsoft_subscription_agrt.pdf

Hello,

I want to split a document containing several invoices into as many files as there are invoices. These are identified by their number.

hear is my code :
def decoupe_fichier(Siret,doc):
#prend un fichier et le découpe en autant de numéro de facture différents
Num_page=0
Num_fact=0
rec=[]
Fournisseur=''
n=Nb_page_traitees(Siret,doc)
New_doc = fitz.open()
sql="SELECT Code_fournisseurs,X0,Y0,X1,Y1 FROM Schema_recup_Data WHERE Nom_colonne='Num_facture' AND Siret='"+ Siret+"'" #récupére le rectangle contenant le num facture
curs1.execute(sql)
for row in curs1:
Fournisseur=row[0]
for pag in doc.pages(0,doc.pageCount, 1): # parcourt le nombre de pages du document ouvert.
Num_fact=page.get_textbox([row[1],row[2],row[3],row[4]])
if pag.get_textbox([row[1],row[2],row[3],row[4]])==Num_fact: #controle par page du num de facture

            Num_page+=1
        else:
            print(pag.number)
            print(pag.get_textbox([row[1],row[2],row[3],row[4]]))
            New_doc.insert_pdf(doc, from_page=pag.number-1, to_page=Num_page-1)
            New_doc.save(path+'/'+Fournisseur+Num_fact+'pdf')
            New_doc.close()

the error returned is :
Traceback (most recent call last):
File "/Users/yves/Recup-Z-interface-compta/BenoitCastel/programme-py/test_extrait_detail_facture_10Aout2021.py", line 322, in
decoupe_fichier(Siret,doc)
File "/Users/yves/Recup-Z-interface-compta/BenoitCastel/programme-py/test_extrait_detail_facture_10Aout2021.py", line 295, in decoupe_fichier
New_doc.insert_pdf(doc, from_page=pag.number-1, to_page=Num_page-1)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/fitz/fitz.py", line 4282, in insert_pdf
raise ValueError("document closed or encrypted")
ValueError: document closed or encrypted

Thanks a lot for your help.
Sincerely.

Restructure the text-extraction folder similarly as with the conventions used in the examples folder.

For the sake of simplicity it may be good idea to merge all demo scripts into the examples folder at once.

the title of the issue says it all.

I dont see the way to use the library converting pdf to png.
This question of mine in SO:
https://stackoverflow.com/questions/69643954/in-python-converting-pdf-to-png-with-python-without-pdf2image?noredirect=1#comment123104887_69643954

got an answer pointing to this library. The solution works but gets results with low resolution.

Hello,
to extract text from an invoice, I use 'blocks' successfully, except for the attached pdf.
My config :
macos BigSur 1.16
pymupdf : 1.19
python : 3.9.7
Thanks for your help.
Sincerely


Facture n°91614287.pdf

I ran this script & found this error. Then, tried to install it using PIP, but there is no module named 'ParseTab' anywhere. Searched Github, nothing! Searched on the Internet, there is absolutely no information about "ParseTab". As a result, the script has become useless. I have latest version of PyMuPDF installed.

Apparently https://github.com/pymupdf/PyMuPDF-Utilities/blob/master/examples/anonymize-document/anonymize.py supports both Python 2 and Python 3, while PyMuPDF dropped Python 2 support quite some time ago. This seems to lead to some issues when running the script itself:

Traceback (most recent call last):
  File "/home/stefan/cwd/redact.py", line 51, in <module>
    stream = remove_txt(stream)
  File "/home/stefan/cwd/redact.py", line 25, in remove_txt
    cont1 = (cont.decode()).replace("\n", " ")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 502: invalid start byte

It is not really clear to me when this example stopped working or whether this affects some documents only. In my case, I switched to byte-strings directly instead of doing any conversions, which seemed to fix it:

def remove_txt(cont):
    """
    Remove everything enclosed in a pair of "BT" / "ET" strings, including both.
    Assuming "cont" is the string of a PDF "/Contents" stream, this will make
    all text of the owning page disappear (permanent delete).
    """
    cont1 = cont.replace(b"\n", b" ")
    ct = cont1.split(b" ")
    nct = []
    intext = False
    for word in ct:
        if word == b"ET":
            intext = False
            continue
        if word == b"BT":
            intext = True
            continue
        if intext:
            continue
        nct.append(word)

    ncont = b" ".join(nct)
    return ncont

are there any method to del the text and pic in pdf?
some examples are needed

in my pdfs there are may same pics and same text words . i want to del them。
but i had not found a def to del text or pic.
help needed.
THANKS!!

`
#coding:utf-8
import fitz,os
rooturl='C:\pdf\'
url='C:\pdf\test.pdf'
doc = fitz.open(url)
imgcount=0
for page in doc:

  imageList = page.get_images()  
  for imginfo in imageList:
      pix = fitz.Pixmap(doc, imginfo[0])
      pix.save(rooturl+"t_{}.png".format(imgcount))
      print(imginfo[0:10])
      doc._deleteObject(imginfo[0])             '''this seems like use xref,maybe i need a way to compare two pic ???'''
      imgcount+=1
    '''above are  all del,maybe there is a way to del by rect,'''


    areas = page.search_for("613950155")
    for area in areas:
        print(area)
        doc._deleteObject(area[0])    '''error'''


    textlist=page.get_text("613950155")
    for textinfo in textlist:
        print(textinfo)
        doc._deleteObject(textinfo)   '''error'''

doc.save('111.pdf')
`

multi_column.py does not identify multiple columns in some cases
2464.pdf

Check in the attached file , in page number 1 where we have clear columns
The utility mix them into one continuous stream
Also page.get_text("text") has the same problem and the stream is mixed

Do lines in the same block means same columns?
Its easy to see lines have a jump from end of one line to the next line in the x-axis

Your configuration (mandatory)

Latest pymupdf

After the first refactor is completed, the examples folder should be refactored into chapters of increasing difficulty like in a book or tutorial:

• 01-the-very-basics
• 02-working-with-text
• 03-working-with-images
• 04-pdf-tools

• Brief description and license
• Usage
• Description (optional)
• Dependencies (without specifying the version)

Will this library supports for python 3+ versions? And I am getting syntax error for do_prompt.

_project.py", line 248
do_prompt(d, "overwrite", "Overwrite existing file? (y/N)", "n", boolean)
^
SyntaxError: invalid syntax

It would be nice to have either some documentation or a requirements file which lists/contains all the addition Python packages necessary to run all of these example scripts. For example I needed PySimpleGUI and Tkinter installed (which I hadn't by default) for running the extract images example.

I tried pdf-converter.py, but i found there maybe some messy code.

left is epub, right is pdf.
when i parsed the pdf,i got the result as the following image.

I tried

python .\fitzcli.py extract -fonts e:\Lab\PyMuPDF-Utilities-master\PyMuPDF-Utilities-master\pdfresource\LUOTUO.pdf -output .\outputfolder

font files save to local file, but i can't open them in windows10's Font Viewer

Good morning,

to identify a supplier, I use an sql query to extract the rectangle containing the supplier identifier.
The rectangle (zone : REC) is stored in a 'blob' zone.
The query returns a string which is not suitable for the page.get_textbox(rec) command which requires rec to be a list

query :
sql_rect="SELECT Vendor_Code,REC FROM Schema_recup_Data WHERE Column_Name='Customer_ID'"
curs1.execute(sql_rect)
for row in curs1:
Identifier=page.get_textbox(row[1])

error returned :
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/fitz/fitz.py", line 903, in init
raise ValueError("bad Rect: sequ. length")
ValueError: bad Rect: sequ. length

What is the solution?

thanks for your feedback.

Best regards.

Translated with www.DeepL.com/Translator (free version)

Make sure that links to examples in Read the Docs won't get broken during and after the restructuring process.

合并带数字签名的发票（比如话费发票）会导致数字印章丢失。

See #88

What is the best way to get better resolution of the final render? I want to get something as comparable to Sumatra reader (as shown in the image), but I can't figure out how to do this. I guess resizing a 2x scaled render is the best way to go about it but I'm not sure how to implement it. Any help would be appreciated :)

Hello. I am using your doc-browser as a lightweight pdf editor. I have added buttons to rotate a page and also reorder the pages but after I click my buttons I don't see the edits because I have to refresh the view. Is there a simple way to refresh the opened PDF?

elif is_Rotate180(btn):
        doc[0].setRotation(90)    
        doc.save(doc.name, incremental = True)
elif is_ReorderPages(btn):
        doc.movePage(1, 0)
        doc.save(doc.name, incremental = True)

Firstly, thanks very much for all the effort behind this library.
I am facing next problem when extracting text from the attached pdf.
BCA2019.pdf

When running this code, I get the title name 'Part A1 Interpreting the NCC' at the end of the html. Meaning, the order of the title is not right.

doc = fitz.open('BCA2019.pdf) 
html = self.doc.get_page_text(14, option='html')

I have been read documentation, and I have go through https://github.com/pymupdf/PyMuPDF-Utilities/tree/master/textbox-extraction
I believe the issue is that the title is inside a rectangle (or something similar).

I will be very grateful if you guide me, in order to extract the text respecting the pdf order.

Thanks in advance.
Regards,
Diego.

I'm trying just a simple example where a couple of words are being replaced, but I can't replace a speficic word because the program doesn't work (the words commented are the ones that broke the code) (I'm trying to find Viviana but I can't). But if I write any other word it does work, that code does run, but if I change text it does not.
What can i do?

import fitz

### READ IN PDF
doc = fitz.open("input.pdf")

for page in doc:
    ### SEARCH
    text = "viiana"
    #text = "viviana"
    #text = "viana"
    text_instances = page.searchFor(text)

    ### HIGHLIGHT
    for inst in text_instances:
        highlight = page.addHighlightAnnot(inst)
        highlight.update()


### OUTPUT
doc.save("output.pdf", garbage=4, deflate=True, clean=True)

Hi!

Thank you for providing Jupyter notebooks as examples 😊
I was able to quickly import one in Google Colab:
https://colab.research.google.com/github/pymupdf/PyMuPDF-Utilities/blob/master/jupyter-notebooks/new-circle-annot.ipynb

I have a couple of suggestions regarding them:

• maybe add this as a first cell, to automatically install PyMuPDF?

!!pip install pymupdf

• maybe add this as the last cell of all notebooks, in order to allow downloading the PDF generated ?

from base64 import b64encode
from IPython.display import display, HTML
base64_pdf = b64encode(doc.tobytes()).decode("utf-8")
# Diplay a download button:
display(HTML(f'<a download="PyMuPDF-example.pdf" href="data:application/pdf;base64,{base64_pdf}">Click to download PDF</a>'))

The picture in PDF has alpha value. So when I extract png from PDF, I get the xref first and then set smask samples value as alpha. It works. However, I found there are several pixel which has different value and color from what we see in PDF. Do you have any ideas or suggestions? Thanks in advance.

tried the find_table jupyter notebook, but can't import fitz in notebook. the error message is as the title. I saw read.me file, it said that pymupdf == 1.23.0 has feature find_table, I also tried in my macbook, but can't install the latest version of package.

Good morning,

please find attached:

• the pdf file with the areas to be extracted,
• the identification of the test area (screen copy)
• the small test python program
• the value returned in the python ide
  I don't understand why the returned text is like this.

Thanks for your help.

Sincerely
Z sorbier janvier 2021.pdf
Capture d’écran 2021-09-27 à 17.06.58.png
Capture d’écran 2021-09-27 à 17.06.23.png

program test python:
import fitz #pymupdf
import sqlite3

path='/Users/yves/Downloads'
source = path+'/Z sorbier janvier 2021.pdf'
conn=sqlite3.connect('/Users/yves/Recup-Z-interface-compta/BenoitCastel/Benoit_Z.db')
curs1=conn.cursor()
sql="SELECT Libel_zone,X0,Y0,Large,Hauteur FROM Parametre_recup_Z where Code_societe='SORBIER'"
curs1.execute(sql)
base_recup = curs1.fetchall() #retourne par rubrique les zones à extraire

Code_societe=''
Periode=''

with fitz.open(source) as doc:
rec=(base_recup[0][1],base_recup[0][2],base_recup[0][1]+base_recup[0][3],base_recup[0][2]+base_recup[0][4])
Code_societe=doc[0].get_textbox(rec)
rec1=(base_recup[1][1],base_recup[1][2],base_recup[1][1]+base_recup[1][3],base_recup[1][2]+base_recup[1][4])
Periode=doc[0].get_textbox(rec1)
rec=(236.2,80.69,301.91,98.29)
print(rec)
Valeur=doc[0].get_textbox(rec)
print(Valeur)

Is it possible to use a font that is already embedded in the PDF? The character to be replaced (space) is already subsetted.

Right now, I'm manually changing the dict of an unpacked PDF and sanitze it afterwards... Doing it in PyMuPDF would save a lot of time.

Traceback (most recent call last):
  File "fitzcli.py", line 1145, in <module>
    main()
  File "fitzcli.py", line 1141, in main
    args.func(args)  # execute requested command
  File "fitzcli.py", line 833, in gettext
    func[args.mode](
  File "fitzcli.py", line 753, in page_layout
    rows = curate_rows(rows, GRID)
  File "fitzcli.py", line 612, in curate_rows
    nrows = [rows[0]]
IndexError: list index out of range

The affected document is private and not permitted for sharing, unfortunately. However, it is a scanned book that consists of many pages with a 2 on 1 layout (left and right side). The OCR layer was added with ocrmypdf.

See #88

Hello there,
I was trying the wxTableExtract example and the pdf was shifting when truing to add a rectangle.
It's working, but the shift makes a part of the pdf disappear behind the menu.

I managed to solve the problem by getting the pdf coordinates
pdfx, pdfy = self.PDFimage.GetPosition()
and adding them to the drawn or updated infos

in functions :
DrawRect --> dc.DrawRectangle(x + pdfx, y + pdfy, w, h)
DrawColumn --> dc.DrawLine(x + pdfx, self.rect_y + pdfy, x + pdfx, self.rect_y + self.rect_h + pdfy)
redraw_bitmap --> dc.DrawBitmap(bm, pdfx, pdfy)

Anyway, thanks again for the work !!!

Good morning,
I am trying the following treatment:

import fitz
import sys
path='/Users/yves/Documents'
filename = path+'/EXTRAIT-05421331612-20211004.pdf'
doc = fitz.open(filename)
test=[]
blocks = doc[0].get_text("blocks")

for block in blocks:
for line in block("lines"):
test.append(valeur)+'/n'

here is the error message I get
I don't understand why.

Python 3.9.7 (v3.9.7:1016ef3790, Aug 30 2021, 16:39:15)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license()" for more information.

============== RESTART: /Users/yves/Documents/test_return_text.py ==============
Traceback (most recent call last):
File "/Users/yves/Documents/test_return_text.py", line 41, in
for line in block("lines"):
TypeError: 'tuple' object is not callable

Thanks for your help.

Sincerely

I have managed to kludge it enough for my purposes. But my changes aren't the "right" thing to do. IF it would help, attached is a diffs file.

I haven't ever used wxPython before, but I can appreciate all the tedious detail work it took to make this example program.

Thank you!

pdfoutline-kludge-diffs.zip

Hi,
on the documentation Release 1.19.0 page 316
we can read
"Function TextPage.extractBLOCKS() (or Page.get_text(“blocks”)) extracts a page’s text blocks as a list of
items like:
(x0, y0, x1, y1, "lines in block", block_type, block_no)" (the page ist joint)

it is rather
(x0, y0, x1, y1, "lines in block", block_no,block_type)

Sincerely
pymupdf-readthedocs-io-en-latest-error.pdf