[Git][reproducible-builds/strip-nondeterminism][master] 6 commits: ar.pm: Add a $member_id to prepare fix for '/' and '//'
Chris Lamb
gitlab at salsa.debian.org
Wed Sep 4 10:38:13 UTC 2019
Chris Lamb pushed to branch master at Reproducible Builds / strip-nondeterminism
Commits:
c83ff4b0 by Marc Herbert at 2019-09-04T10:37:31Z
ar.pm: Add a $member_id to prepare fix for '/' and '//'
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
14f88248 by Marc Herbert at 2019-09-04T10:37:33Z
ar.pm: Don't corrupt the pseudo-filemode of the symbols table '/'
Don't reset to "644" the pseudo-permissions of the symbols table '/'. Whatever
the placeholder is ("0" with GNU ar 2.31), messing with it diverges with the
output of GNU ar which is already deterministic.
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
3943c5e2 by Marc Herbert at 2019-09-04T10:37:34Z
ar.pm: Stop corrupting the SystemV/GNU table of long filenames
Long filenames are stored in a special section with a completely different
format. Stop assuming it has regular metadata and stop corrupting it.
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
f54aa08f by Marc Herbert at 2019-09-04T10:37:35Z
ar.pm: Add test files for symtab ('/') and long names ('//')
Note symbol_table.out cannot be produced by GNU ar alone because it can't set
any specified timestamp on the symbol table.
long_member_name.a.out cannot be created by GNU ar alone either for a
completely different, strange and permission-related reason. Faking a
specified timestamp for archive members is easy pre-archiving with "touch -r".
However this obviously requires 'ar U'. For long filenames, ar U maps
permissions 644 to the string "100644" for some unknown reason. Both ar D and
strip-nondeterminism use the permission string "644" (displayed by diffoscope
as "?rw-r--r--")
This is the script which was used to create these test files:
create_ar_testcase()
{
ar_base="$1"; shift
for o in "$@"; do
objcopy --remove-section=.comment \
--remove-section=.note.GNU-stack \
--remove-section=.eh_frame \
"$o" || true
done
touch "$@"
rm -f "${ar_base}".a.in "${ar_base}".a.out "${ar_base}".a.D
ar qU "${ar_base}".a.in "$@"
ar qD "${ar_base}".a.D "$@"
cp "${ar_base}".a.D "${ar_base}".a
strip-nondeterminism --normalizers +all -v -T 1423159771 "${ar_base}".a
mv "${ar_base}".a "${ar_base}".a.out
}
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
7a88b046 by Marc Herbert at 2019-09-04T10:37:36Z
Add new $File::StripNondeterminism::verbose global
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
a9824cd6 by Marc Herbert at 2019-09-04T10:37:38Z
ar.pm: Tell user that GNU ar could not set the symbol table's mtime
... in verbose mode, when setting it.
In other words tell the user when GNU ar cannot reproduce the output of
strip-nondeterminism.
Signed-off-by: Chris Lamb <lamby at debian.org>
- - - - -
11 changed files:
- bin/strip-nondeterminism
- + debian/source/include-binaries
- lib/File/StripNondeterminism.pm
- lib/File/StripNondeterminism/handlers/ar.pm
- t/fixtures.t
- + t/fixtures/ar/long_member_name.a.in
- + t/fixtures/ar/long_member_name.a.out
- + t/fixtures/ar/longnames_symtab.a.in
- + t/fixtures/ar/longnames_symtab.a.out
- + t/fixtures/ar/symbol_table.a.in
- + t/fixtures/ar/symbol_table.a.out
Changes:
=====================================
bin/strip-nondeterminism
=====================================
@@ -84,6 +84,7 @@ for (split(/,/, $normalizers // '')) {
$File::StripNondeterminism::canonical_time = $timestamp;
$File::StripNondeterminism::clamp_time = $clamp_timestamp;
+$File::StripNondeterminism::verbose = $verbose;
die $usage if @ARGV == 0;
=====================================
debian/source/include-binaries
=====================================
@@ -0,0 +1,6 @@
+t/fixtures/ar/long_member_name.a.in
+t/fixtures/ar/long_member_name.a.out
+t/fixtures/ar/longnames_symtab.a.in
+t/fixtures/ar/longnames_symtab.a.out
+t/fixtures/ar/symbol_table.a.in
+t/fixtures/ar/symbol_table.a.out
=====================================
lib/File/StripNondeterminism.pm
=====================================
@@ -23,7 +23,7 @@ use warnings;
use POSIX qw(tzset);
-our($VERSION, $canonical_time, $clamp_time);
+our($VERSION, $canonical_time, $clamp_time, $verbose);
$VERSION = '1.5.0'; # <https://semver.org/>
=====================================
lib/File/StripNondeterminism/handlers/ar.pm
=====================================
@@ -65,15 +65,35 @@ sub normalize {
die "Incorrect file magic"
if substr($buf, 58, length($FILE_MAGIC)) ne $FILE_MAGIC;
+ # $member_id is the member's filename if it's short
+ # enough to fit in 16 characters. Otherwise it's a
+ # "/number" index in the table of long member names '//'
+ # (SysV/GNU), or a #1/ prefixed length (BSD)
+ my $member_id = substr($buf, 0, 16);
+
my $file_mode = oct(substr($buf, 40, 8));
my $file_size = substr($buf, 48, 10);
die "Incorrect file size"
if $file_size < 1;
+ # Don't touch the System V/GNU table of long filenames
+ # '//', it's a different format and already
+ # deterministic.
+ if (substr($member_id, 0, 3) eq "// ") {
+ goto NEXT_MEMBER;
+ }
+
seek $fh, $file_header_start + 16, SEEK_SET;
# mtime
+ if ($File::StripNondeterminism::verbose
+ && $File::StripNondeterminism::canonical_time
+ && substr($member_id, 0, 2) eq "/ ") {
+ print "Setting symbols table's mtime in $file to: "
+ . gmtime($File::StripNondeterminism::canonical_time)
+ . ". GNU ar cannot do this.\n";
+ }
syswrite $fh,
sprintf("%-12d", $File::StripNondeterminism::canonical_time // 0);
# owner
@@ -81,10 +101,15 @@ sub normalize {
# group
syswrite $fh, sprintf("%-6d", 0);
# file mode
+
+ # Don't touch the pseudo-"filemode" of the symbols table '/ '
+ if (substr($member_id, 0, 2) eq "/ ") {
+ goto NEXT_MEMBER;
+ }
syswrite $fh,
sprintf("%-8o", ($file_mode & oct(100)) ? oct(755) : oct(644));
- # move to next member
+ NEXT_MEMBER:
my $padding = $file_size % 2;
seek $fh,
$file_header_start + $FILE_HEADER_LENGTH + $file_size + $padding,
=====================================
t/fixtures.t
=====================================
@@ -53,6 +53,7 @@ for (File::StripNondeterminism::all_normalizers()) {
File::StripNondeterminism::enable_normalizer($_);
}
+# = 2015-02-05 10:09:31
$File::StripNondeterminism::canonical_time = 1423159771;
my @fixtures = glob('t/fixtures/*/*.in');
=====================================
t/fixtures/ar/long_member_name.a.in
=====================================
@@ -0,0 +1,6 @@
+!<arch>
+// 28 `
+filename_longer_than16.txt/
+/0 1423159771 1001 0 100664 45 `
+a file with a name longer than 16 characters
+
=====================================
t/fixtures/ar/long_member_name.a.out
=====================================
@@ -0,0 +1,6 @@
+!<arch>
+// 28 `
+filename_longer_than16.txt/
+/0 1423159771 0 0 644 45 `
+a file with a name longer than 16 characters
+
=====================================
t/fixtures/ar/longnames_symtab.a.in
=====================================
Binary files /dev/null and b/t/fixtures/ar/longnames_symtab.a.in differ
=====================================
t/fixtures/ar/longnames_symtab.a.out
=====================================
Binary files /dev/null and b/t/fixtures/ar/longnames_symtab.a.out differ
=====================================
t/fixtures/ar/symbol_table.a.in
=====================================
Binary files /dev/null and b/t/fixtures/ar/symbol_table.a.in differ
=====================================
t/fixtures/ar/symbol_table.a.out
=====================================
Binary files /dev/null and b/t/fixtures/ar/symbol_table.a.out differ
View it on GitLab: https://salsa.debian.org/reproducible-builds/strip-nondeterminism/compare/e1def588d2d1f8859343db65dddd19815dad1c51...a9824cd69f587841ce4c5746dd475102228f8ca7
--
View it on GitLab: https://salsa.debian.org/reproducible-builds/strip-nondeterminism/compare/e1def588d2d1f8859343db65dddd19815dad1c51...a9824cd69f587841ce4c5746dd475102228f8ca7
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20190904/927dd26d/attachment.html>
More information about the rb-commits
mailing list