page.get_pixmap missing text for some live text pdf.

[nissansz]

def get_pdf_page_image(page, zoom=2):

mat = fitz.Matrix(zoom, zoom)
pix = page.get_pixmap(matrix=mat, colorspace=fitz.csRGB)
img_bytes = pix.tobytes("png")
return io.BytesIO(img_bytes)

original pdf live text

pptx image from page.get_pixmap

[JorjMcKie]

Please provide the original file / page for reproduction

[JorjMcKie]

Don't see your problem: when executing CLI mutool extraction a lot of warnings appear:

$ mutool convert -o p2-mutool.png p2.pdf
warning: assert: too many code space ranges
warning: ... repeated 60 times...
warning: cannot encode character
warning: ... repeated 24 times...
warning: assert: too many code space ranges
warning: ... repeated 32 times...
warning: cannot encode character
warning: ... repeated 56 times...
warning: assert: too many code space ranges
warning: ... repeated 18 times...
warning: cannot encode character
warning: ... repeated 18 times...

However a PNG is generated that looks like the the pdf.

[nissansz]

Your png has same problem, missing text

[nissansz]

Still showing image, please delete it.

Don't see your problem: when executing CLI mutool extraction a lot of warnings appear:

$ mutool convert -o p2-mutool.png p2.pdf
warning: assert: too many code space ranges
warning: ... repeated 60 times...
warning: cannot encode character
warning: ... repeated 24 times...
warning: assert: too many code space ranges
warning: ... repeated 32 times...
warning: cannot encode character
warning: ... repeated 56 times...
warning: assert: too many code space ranges
warning: ... repeated 18 times...
warning: cannot encode character
warning: ... repeated 18 times...

However a PNG is generated that looks like the the pdf.

[JorjMcKie]

MeiryoUI‑OIT with WinAnsiEncoding cannot address CJK characters

This is the font used for your problem glyph examples.

Even though Meiryo UI is a fully Unicode‑capable Japanese font, the way it is embedded in your PDF prevents it from displaying any CJK characters. When the font is embedded using WinAnsiEncoding, only the first 256 codepoints (0–255) are available. This encoding covers Western Latin characters only.

CJK characters such as 質, 術, and 規 lie in the Unicode range U+4E00–U+9FFF, far outside the WinAnsi range. Because of this:

The PDF producer embeds only the Western glyphs from MeiryoUI‑OIT

The CJK glyphs are not embedded at all

The PDF contains no usable glyphs for those characters in this font

As a result, MuPDF has no glyph data to render and shows nothing. Other PDF viewers compensate by performing automatic font fallback, but MuPDF never does that.

[nissansz]

So no solution with MuPDF, right?

Please delete image in your comment since it is personal file.

[JorjMcKie]

So no solution with MuPDF, right?

Please delete above image since it is personal file.

Correct.
Let me try to remove your data ...

[nissansz]

Thank you.
I have another question for Korean.
Some text is rendered as drawings in pdf, but there is a normal text layer above them, any method to convert those text to a normal text layer only and delete drawing text?
Attached original P3.pdf, and extracted groups of text, images, drawings,

[JorjMcKie]

You can use redactions to achieve this:
Insert redaction annotations that cover the required rectangles - in this case probably the the full page: page.rect

The execute page.apply_redactions(images=imgopt, graphics=vecopt, text=textopt).

Give each of the parameters (integers) the desired value:

import pymupdf
help(pymupdf.Page.apply_redactions)
Help on function apply_redactions in module pymupdf:

apply_redactions(
    page: 'Page',
    images: int = 2,
    graphics: int = 1,
    text: int = 0
) -> bool
    Apply the redaction annotations of the page.

    Args:
        page: the PDF page.
        images:
              0 - ignore images
              1 - remove all overlapping images
              2 - blank out overlapping image parts
              3 - remove image unless invisible
        graphics:
              0 - ignore graphics
              1 - remove graphics if contained in rectangle
              2 - remove all overlapping graphics
        text:
              0 - remove text
              1 - ignore text

Each object type can be removed or kept independently.

[nissansz]

I tried, some graphics may contain needed images, they are removed too with apply_redactions.

Don't know if any method to recognize text only graphics?

[JorjMcKie]

Don't know if any method to recognize text only graphics?

No, there is no way to do this.

[JorjMcKie]

Such vectors do have a couple of characteristics that make them probable or plausible candidates for resembling characters like

• wrapping rectangle has typical character dimensions: width <= 15, height <= 15
• Bézier curves and non-axis-parallel lines are part of the path

But there also are counter-examples

[nissansz]

Any method to check if a text block/line is visible?
If invisible, set them to visible?

[JorjMcKie]

• Look at the text spans: if the char_flags bits indicate non-stroke and non-fill (look up documentation), then the characters are not visible. This is the typical case for members of OCR text layers.
• If span alpha = 0 and no Type3 font: invisible
• Then the wide area of cases black-on-black / white-on-white text. Check out the Pixmap colors of the bbox: if the dominating color equals the text color: invisible.
• Then the wide area of text covered by something else: no direct way.

If invisible, set them to visible?

No way...

[nissansz]

All text span are grouped to visible text pdf.
But in actual pdf, some text above drawing text is invisible.

{'size': 9.960000038146973, 'flags': 4, 'font': 'CJONLYONENEWbodyLight', 'color': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': '2', 'origin': (768.6699829101562, 532.6320190429688), 'bbox': (768.6699829101562, 523.1600341796875, 774.207763671875, 535.6199951171875)}
{'size': 21.959999084472656, 'flags': 20, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': '低투자기반의북미사업확대, 시너지기반시장블록화, 정밀발효기술기반', 'origin': (69.76799774169922, 77.1400146484375), 'bbox': (69.76799774169922, 56.25605773925781, 710.6491088867188, 83.72801208496094)}
{'size': 21.983999252319336, 'flags': 20, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'Biofoundary 미래사업기반확보목적A社와의JV 추진위해실사진행中', 'origin': (52.007999420166016, 99.10000610351562), 'bbox': (52.007999420166016, 78.19322204589844, 728.5574340820312, 105.69520568847656)}
{'size': 24.0, 'flags': 20, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'I. 사업장Overview ', 'origin': (11.640000343322754, 38.17999267578125), 'bbox': (11.640000343322754, 15.355993270874023, 210.40798950195312, 45.3799934387207)}
{'size': 12.02400016784668, 'flags': 20, 'font': 'CJONLYONENEWtitleBold', 'color': 16777215, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'Decatur, Illinois', 'origin': (515.760009765625, 148.20001220703125), 'bbox': (515.760009765625, 136.7651824951172, 615.1986083984375, 151.8072052001953)}
{'size': 8.399999618530273, 'flags': 0, 'font': 'ArialMT', 'color': 9605778, 'ascender': 0.9052734375, 'descender': -0.2119140625, 'text': '►', 'origin': (498.1199951171875, 180.98001098632812), 'bbox': (498.1199951171875, 173.37571716308594, 506.4338684082031, 182.7600860595703)}

[JorjMcKie]

My above list doesn't claim to be complete, there are things like clipped text, ActualText and what not else.

[nissansz]

What is meaning of 'flags': 4,? Any list of definitions for flags?

[JorjMcKie]

See here https://pymupdf.readthedocs.io/en/latest/textpage.html#structure-of-dictionary-outputs

Your PyMuPDF is not up to date.

[nissansz]

It seems alpha==0 is the target, if I only want to set all alpha ==0 text to 255, and keep all font details like italic, bold, space between Korean characters, etc., retain all other images, graphics, textboxe, what is the best method?

{'size': 9.960000038146973, 'flags': 4, 'bidi': 0, 'char_flags': 16, 'font': 'CJONLYONENEWbodyLight', 'color': 0, 'alpha': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': '2', 'origin': (768.6699829101562, 532.6320190429688), 'bbox': (768.6699829101562, 523.1600341796875, 774.207763671875, 535.6199951171875)}
{'size': 21.959999084472656, 'flags': 20, 'bidi': 0, 'char_flags': 24, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'alpha': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': '低투자기반의북미사업확대, 시너지기반시장블록화, 정밀발효기술기반', 'origin': (69.76799774169922, 77.1400146484375), 'bbox': (69.76799774169922, 56.25605773925781, 710.6491088867188, 83.72801208496094)}
{'size': 21.983999252319336, 'flags': 20, 'bidi': 0, 'char_flags': 24, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'alpha': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'Biofoundary 미래사업기반확보목적A社와의JV 추진위해실사진행中', 'origin': (52.007999420166016, 99.10000610351562), 'bbox': (52.007999420166016, 78.19322204589844, 728.5574340820312, 105.69520568847656)}
{'size': 24.0, 'flags': 20, 'bidi': 0, 'char_flags': 24, 'font': 'CJONLYONENEWtitleBold', 'color': 0, 'alpha': 0, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'I. 사업장Overview ', 'origin': (11.640000343322754, 38.17999267578125), 'bbox': (11.640000343322754, 15.355993270874023, 210.40798950195312, 45.3799934387207)}
{'size': 12.02400016784668, 'flags': 20, 'bidi': 0, 'char_flags': 24, 'font': 'CJONLYONENEWtitleBold', 'color': 16777215, 'alpha': 255, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': 'Decatur, Illinois', 'origin': (515.760009765625, 148.20001220703125), 'bbox': (515.760009765625, 136.7651824951172, 615.1986083984375, 151.8072052001953)}
{'size': 8.399999618530273, 'flags': 0, 'bidi': 0, 'char_flags': 16, 'font': 'ArialMT', 'color': 8421504, 'alpha': 255, 'ascender': 0.9049999713897705, 'descender': -0.20999999344348907, 'text': '►', 'origin': (498.1199951171875, 180.98001098632812), 'bbox': (498.1199951171875, 173.3780059814453, 506.4360046386719, 182.74400329589844)}
{'size': 12.0, 'flags': 4, 'bidi': 0, 'char_flags': 16, 'font': 'CJONLYONENEWbodyRegular', 'color': 0, 'alpha': 255, 'ascender': 0.9509999752044678, 'descender': -0.30000001192092896, 'text': '옥수수 가공, 바이오에탄올 사업 중심으로 사료용 ', 'origin': (512.52001953125, 180.98001098632812), 'bbox': (512.52001953125, 169.56800842285156, 744.2899780273438, 184.58001708984375)}

[nissansz]

found a method to edit xref for alpha to 255, it seems working

[nissansz]

xref alpha does not help much,
it seems necessary to rewrite all text for alpha==0, any method to set bold/italic/underline from span during write?