DMF7

[blaineoliver]

Hi, does shift support DMF7 with its offline command?

[pkolano]

We don't have DMF7 in production yet, so it hasn't been incorporated and I haven't yet looked into the differences. I believe we may have a DMF7 test system around, so I'll try to take a look to see what support entails.

[pkolano]

Turns out we don't have a test system yet. I did look at the admin guide and it seems like there are some fairly significant changes so wouldn't feel comfortable adding support until I have a system to test on. Can revisit once we have such a system.

[blaineoliver]

Thanks for the input, do you know why when compiling I am getting the following error?

root@c4n1541.hpc4:[c]# make lustre
gcc -o shift-bin -g shift-bin.c -lattr -lacl -llustreapi
shift-bin.c: In function ‘do_fallocate’:
shift-bin.c:132:28: error: ‘FALLOC_FL_KEEP_SIZE’ undeclared (first use in this function)
int rc = fallocate(fd, FALLOC_FL_KEEP_SIZE, 0, len);
^
shift-bin.c:132:28: note: each undeclared identifier is reported only once for each function it appears in
shift-bin.c: In function ‘do_getstripe’:
shift-bin.c:231:44: error: ‘LLAPI_LAYOUT_COMP_USE_FIRST’ undeclared (first use in this function)
int rc = llapi_layout_comp_use(layout, LLAPI_LAYOUT_COMP_USE_FIRST);
^
shift-bin.c:241:44: error: ‘LLAPI_LAYOUT_COMP_USE_NEXT’ undeclared (first use in this function)
rc = llapi_layout_comp_use(layout, LLAPI_LAYOUT_COMP_USE_NEXT);
^
make: *** [lustre] Error 1

I already had to add libacl-devel and this solved my first issue, but this issue I am unsure of.

[pkolano]

Which version of lustre are you using (via "lfs --version")? It should work with 2.12. The API did change between some earlier versions (don't remember which) and it's possible it changed again on 2.14. If you are on an earlier release than 2.12, you can try the previous release of the shift-bin.c file, which was for the previous API. Otherwise, you can just build the nolustre version. With more recent versions, the striping stuff is not as critical with the addition of the progressive file layout feature. shift-bin is also optional so it will just the lfs command if not present, although file system traversal stage is much faster with shift-bin. Unfortunately, it does not detect if shift-bin has lustre support or not, so striping will be a noop if it is present, but built without lustre support.

Also, just spoke with HPE engineers about DMF7 and there are some aspects currently missing from DMF7 that need to be addressed before shift can support it. Currently, there is no way to pipe file lists into "dmf get" or "dmf put", but a comparable mechanism is supposed to be added at some point. It sounds like there may also be complications from new authentication mechanisms.