Also attempt to attain the value of TEXMF using kpse library

[RadioNoiseE]

Hi!

I am installing digestif for my self-built TeX distribution which does not install kpsewhich, and encountered an issue of not being able to run kpsewhich.

After inspecting I found that it's because of this logic in config.lua which tries to find texmf root using kpsewhich:

digestif/digestif/config.lua

Lines 12 to 31 in f2b6059

	if util.is_command("kpsewhich") then
	local pipe = io.popen("kpsewhich -var-brace-value=TEXMF")
	local output = pipe:read("l")
	local ok, exitt, exitc = pipe:close()
	if ok and exitt == "exit" and exitc == 0 then
	config.texmf_dirs = util.imap(
	function (s) return s:gsub("^!!", "") end,
	util.path_list_split(output)
	)
	elseif config.verbose then
	util.log("Error running kpsewhich (%s %d)", exitt, exitc)
	end
	else -- TODO: What should be the default?
	config.texmf_dirs = {
	"/usr/local/share/texmf",
	"/usr/share/texmf",
	"/usr/share/texlive/texmf-local",
	"/usr/share/texlive/texmf-dist",
	}
	end

I am aware that this should be compatible with non-texlua Lua interpreters, however I think it would be a good idea to try kpse.var_value after execution failed.

<string> r = kpse.var_value(<string> s) -- similar to kpsewhich’s -var-value

Thanks for this awesome project!

[astoff]

Hah, I went to read those lines and realized I already had an unpushed changed related to the issue you mentioned. Do you want to test the version in #78?

(PS: As an alternative you can always set DIGESTIF_TEXMF, but of course making the automatic discovery work is best.)

[RadioNoiseE]

I've tested this, and after some modifications it worked.

First issue: there are three valid lua_type

One workaround is:

local lua_engines = {
  luatex = true,
  luahbtex = true,
  luajithbtex = true,
}
...
elseif lua_engines[lua_type] then
...

Second issue: use valid program (binary file) name for kpse.new

See the screenshot, initializing kpathsea with tex will not work.

local kpathsea = _G.kpse.new("digestif")

Third issue: typo in #config.texmf_dirs

if #config.textmf_dirs == 0 then
              ^

[astoff]

I've pushed some more changes, it will probably work for you now.

[RadioNoiseE]

Unfortunately it doesn't.

It complains about:

warning: kpathsea: configuration file texmf.cnf not found in these directories: /Users/jing/web2c.
/usr/local/bin/digestif:11: Can't find 'digestif.zip' archive

which is fixed after changing kpse.set_program_name("","tex") to kpse.set_program_name("luatex") or kpse.set_program_name("digestif")

[astoff]

Hum... So, based on the documentation and not on experimentation, I thought kpse.set_program_name("digestif") would not crash but also not find anything. But apparently that's not the case :-).

How about passing the empty string (as in the current version of the PR)?

What I don't like about passing a non-empty string is that for whatever reason kpse will search for a program of that name, and who knows when things are not installed with a different name... Thanks for helping nailing this down.

[RadioNoiseE]

It seems reasonable to use the name of the currently running executable--assuming Lua can retrieve it.

Will check whether providing an empty string to kpse.new functions correctly.

[RadioNoiseE]

How about passing the empty string (as in the current version of the PR)?

Looks like passing an empty string doesn't work.

According to my understanding, setting an invalid (non-existent) <string> name will inevitably break things like $SELFAUTODIR

Here's my TEXMF to help make things clearer

jing@rne ~ % cat /usr/local/lib/texmf/web2c/texmf.cnf 
% ---------------------------
% Search Path and Directories
% ---------------------------
TEXMFROOT     = $SELFAUTOPARENT
TEXMFDIST     = $TEXMFROOT/texmf
TEXMFHOME     = ~/.texmf
TEXMFVAR      = $TEXMFHOME/cache
TEXMF         = {$TEXMFHOME,$TEXMFDIST}
TEXMFCACHE    = $TEXMFVAR
WEB2C         = $TEXMF/web2c
TEXMFDOTDIR   = .
TEXINPUTS     = $TEXMFDOTDIR;$TEXMF/generic//
LUAINPUTS     = $TEXINPUTS
CLUAINPUTS    = $LUAINPUTS
MFINPUTS      = $TEXMFDOTDIR;$TEXMF/metafont//
MPINPUTS      = $TEXMFDOTDIR;$TEXMF/metapost//
TEXFORMATS    = $TEXMFDOTDIR;$TEXMF/web2c{/$engine,}
TEXFONTMAPS   = $TEXMFDOTDIR;$TEXMF/fonts/map/{$progname,}//
OSFONTDIR     = {/System,}/Library/Fonts//
T1FONTS       = $TEXMFDOTDIR;$TEXMF/fonts/type1//;$OSFONTDIR//
TTFONTS       = $TEXMFDOTDIR;$TEXMF/fonts/truetype//;$OSFONTDIR//
OPENTYPEFONTS = $TEXMFDOTDIR;$TEXMF/fonts/opentype//;$OSFONTDIR//
OFMFONTS      = $TEXMFDOTDIR;$TEXMF/fonts/{ofm,tfm}//
TFMFONTS      = $OFMFONTS
OVFFONTS      = $TEXMFDOTDIR;$TEXMF/fonts/{ovf,vf}//
VFFONTS       = $OVFFONTS
TEXMFCNF      = $SELFAUTODIR/web2c
TEXMFOUTPUT   = /tmp
MISSFONT_LOG  = missfont.log

What I don't like about passing a non-empty string is that for whatever reason kpse will search for a program of that name, and who knows when things are not installed with a different name...

Using arg[0]:match("([^/]+)$") instead of the empty string seems to be an acceptable solution? The below modification works flawlessly.

-- in bin/digestif.texlua line 8
local archive = kpse.new(arg[0]:match("([^/]+)$")):find_file("digestif.zip", "other binary files")

-- and in digestif/config.lua line 113
local ok, kpse = pcall(function() return _G.kpse.new(arg[0]:match("([^/]+)$")) end)

My understanding here is that, once Kpathsea find the executable, $SELFAUTODIR will be properly set, and texmf.cnf found, then all other paths are being expanded and defined.

Thanks for helping nailing this down.

Thanks for working on this issue :)

[astoff]

That's funny, how come kpse.new("") works for me? How about kpse.new(".")?

Anyway, using arg[0] instead of that substring should also work. It seems that all that matters is that a readable file is passed as argument. I've updated the pull request with this variant.

[RadioNoiseE]

That's funny, how come kpse.new("") works for me? How about kpse.new(".")?

I believe it's because the TeX distribution you use does not use things like $SELFAUTODIR in the texmf.cnf and texmf.in (compile time). Both don't work for me.

Anyway, using arg[0] instead of that substring should also work. It seems that all that matters is that a readable file is passed as argument.

Not really. I'm afraid it does not. Compare the case when arg[0] is used

jing@rne digestif % texlua --kpathsea-debug=3 /usr/local/lib/texmf/web2c/digestif
kdebug:hash_lookup(TEXMFDBS./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFDBS) => (nil)
kdebug:hash_lookup(TEXMFLOG./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFLOG) => (nil)
kdebug:hash_lookup(/USR/LOCAL/LIB/TEXMF/WEB2C/DIGESTIFINPUTS./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(/USR/LOCAL/LIB/TEXMF/WEB2C/DIGESTIFINPUTS) => (nil)
kdebug:hash_lookup(TEXMF./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMF) => {$TEXMFHOME,$TEXMFDIST}
kdebug:hash_lookup(TEXMFHOME./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFHOME) => ~/.texmf
kdebug:hash_lookup(TEXMFDIST./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFDIST) => $TEXMFROOT/texmf
kdebug:hash_lookup(TEXMFROOT./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFROOT) => $SELFAUTOPARENT
kdebug:hash_lookup(try_std_extension_first./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(try_std_extension_first) => t
kdebug:hash_lookup(texmf_casefold_search./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(texmf_casefold_search) => 1
kdebug:hash_lookup(/Users/jing/.texmf/cache) => (nil)
kdebug:hash_lookup(/Users/jing/.texmf/cache/luamplib_cache) => (nil)
kdebug:hash_lookup(/Users/jing/.texmf/cache/luamplib_cache/luamplib_input_hatching_mp) => (nil)
kdebug:hash_lookup(/Users/jing/.texmf/cache/luatexja) => (nil)
kdebug:hash_lookup(/Users/jing/.texmf/cache/luatexja/ltj-jisx0208.luc) => (nil)
kdebug:hash_lookup(/Users/jing/.texmf/cache/luatexja/extra_sourcehansanssc-regular.lua.gz) => (nil)
..................
failed attempts...
..................
kdebug:hash_lookup(/usr/local/lib/texmf/fonts/map) => (nil)
kdebug:hash_lookup(/usr/local/lib/texmf/fonts/map/luatex.map) => (nil)
/usr/local/lib/texmf/web2c/digestif:9: Can't find 'digestif.zip' archive

to when arg[0]:match("([^/]+)$") is used:

jing@rne digestif % texlua --kpathsea-debug=3 /usr/local/lib/texmf/web2c/digestif
kdebug:hash_lookup(TEXMFDBS.digestif) => (nil)
kdebug:hash_lookup(TEXMFDBS) => (nil)
kdebug:hash_lookup(TEXMFLOG.digestif) => (nil)
kdebug:hash_lookup(TEXMFLOG) => (nil)
kdebug:hash_lookup(DIGESTIFINPUTS.digestif) => (nil)
kdebug:hash_lookup(DIGESTIFINPUTS) => (nil)
kdebug:hash_lookup(TEXMF.digestif) => (nil)
kdebug:hash_lookup(TEXMF) => {$TEXMFHOME,$TEXMFDIST}
kdebug:hash_lookup(TEXMFHOME.digestif) => (nil)
kdebug:hash_lookup(TEXMFHOME) => ~/.texmf
kdebug:hash_lookup(TEXMFDIST.digestif) => (nil)
kdebug:hash_lookup(TEXMFDIST) => $TEXMFROOT/texmf
kdebug:hash_lookup(TEXMFROOT.digestif) => (nil)
kdebug:hash_lookup(TEXMFROOT) => $SELFAUTOPARENT
kdebug:hash_lookup(try_std_extension_first.digestif) => (nil)
kdebug:hash_lookup(try_std_extension_first) => t
kdebug:hash_lookup(texmf_casefold_search.digestif) => (nil)
kdebug:hash_lookup(texmf_casefold_search) => 1
kdebug:hash_lookup(/usr/local/lib/texmf/digestif/digestif.zip) => (nil)
kdebug:hash_lookup(TEXMFDBS./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFDBS) => (nil)
kdebug:hash_lookup(TEXMFLOG./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFLOG) => (nil)
kdebug:hash_lookup(TEXMF./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMF) => {$TEXMFHOME,$TEXMFDIST}
kdebug:hash_lookup(TEXMFHOME./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFHOME) => ~/.texmf
kdebug:hash_lookup(TEXMFDIST./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFDIST) => $TEXMFROOT/texmf
kdebug:hash_lookup(TEXMFROOT./usr/local/lib/texmf/web2c/digestif) => (nil)
kdebug:hash_lookup(TEXMFROOT) => $SELFAUTOPARENT

^C

OFC leaving name as is and set the progname to the basename should also work

[RadioNoiseE]

Just to be clear, the modification to the PR that will make kpse work is changing arg[0] to arg[0]:match("([^/]+)$")

[astoff]

What I don't like about using arg[0]:match("([^/]+)$") is that it wouldn't work if that program name is not in PATH. This is not an entirely inconceivable scenario, as the user doesn't really need to use digestif directly. All that's needed is to set up a text editor to find that program, possibly by supplying the full path.

So I think changing it to kpse.new("texlua") is the best option, as the shebang already assumes that program can be found in the PATH (#!/usr/bin/env texlua).

Why is this so confusing? 😅

[RadioNoiseE]

So I think changing it to kpse.new("texlua") is the best option, as the shebang already assumes that program can be found in the PATH (#!/usr/bin/env texlua).

I agree.

Why is this so confusing? 😅

Kpathsea is some 50 years+ old software :)

[astoff]

I assume you mean it works for your now, so I merged the PR. Thanks again and let me know if you have more issues.