[Git][reproducible-builds/strip-nondeterminism][debian] 6 commits: Correct indentation level of .zip file field format comment.

[Chris Lamb]

Chris Lamb pushed to branch debian at Reproducible Builds / strip-nondeterminism


Commits:
a553d343 by Chris Lamb at 2019-01-29T14:22:39Z
Correct indentation level of .zip file field format comment.

- - - - -
3f4ba2fb by Chris Lamb at 2019-01-29T14:40:31Z
Clarify negative message in test failures.

- - - - -
23d02adf by Chris Lamb at 2019-02-13T08:22:06Z
dh_strip_nondeterminism: Deduplicate hardlinks via stat(2) to avoid issues when processing files in parallel. As strip-nondeterminism's handlers are not currently guaranteed to be atomic, one process can temporarily truncate a file which can cause errors in other threads processing the "same" file under a unique pathname. This was causing a FTBFS in packages that deduplicate hardlinks in their build process (eg. src:debian-handbook). (Closes: #922168)

- - - - -
1ccc8241 by Chris Lamb at 2019-02-13T08:29:48Z
Release version 1.1.1

- - - - -
4b8155dc by Chris Lamb at 2019-02-13T08:30:52Z
Merge tag '1.1.1' into debian

Release version 1.1.1

* tag '1.1.1':
  Release version 1.1.1
  dh_strip_nondeterminism: Deduplicate hardlinks via stat(2) to avoid issues when processing files in parallel. As strip-nondeterminism's handlers are not currently guaranteed to be atomic, one process can temporarily truncate a file which can cause errors in other threads processing the "same" file under a unique pathname. This was causing a FTBFS in packages that deduplicate hardlinks in their build process (eg. src:debian-handbook). (Closes: #922168)
  Clarify negative message in test failures.
  Correct indentation level of .zip file field format comment.

- - - - -
f360b3ce by Chris Lamb at 2019-02-13T08:32:03Z
Update changelog for 1.1.1-1 release

- - - - -


5 changed files:

- bin/dh_strip_nondeterminism
- debian/changelog
- lib/File/StripNondeterminism.pm
- lib/File/StripNondeterminism/handlers/zip.pm
- t/fixtures.t


Changes:

=====================================
bin/dh_strip_nondeterminism
=====================================
@@ -45,7 +45,7 @@ things to exclude.
 
 init();
 
-my @nondeterministic_files;
+my (@nondeterministic_files, %seen);
 
 sub testfile {
 	return if -l $_ or -d $_; # Skip directories and symlinks always.
@@ -57,6 +57,11 @@ sub testfile {
 		return if ($fn=~m/\Q$f\E/);
 	}
 
+	# Deduplicate hardlinks to avoid issues under parallelism
+	my ($dev, $inode, undef, $nlink) = stat($_);
+	return if defined $nlink && $nlink > 1 && $seen{"$inode.$dev"};
+	$seen{"$inode.$dev"} = 1;
+
 	my $normalizer = File::StripNondeterminism::get_normalizer_for_file($_);
 	if ($normalizer) {
 		push @nondeterministic_files, [$fn, $normalizer];


=====================================
debian/changelog
=====================================
@@ -1,3 +1,16 @@
+strip-nondeterminism (1.1.1-1) unstable; urgency=medium
+
+  * dh_strip_nondeterminism: Deduplicate hardlinks via stat(2) to avoid issues
+    when processing files in parallel.
+
+    As strip-nondeterminism's handlers are not currently guaranteed to be
+    atomic, one process can temporarily truncate a file which can cause errors
+    in other threads processing the "same" file under a unique pathname. This
+    was causing a FTBFS in packages that deduplicate hardlinks in their build
+    process (eg. src:debian-handbook). (Closes: #922168)
+
+ -- Chris Lamb <lamby at debian.org>  Wed, 13 Feb 2019 09:31:59 +0100
+
 strip-nondeterminism (1.1.0-1) unstable; urgency=medium
 
   [ Holger Levsen ]


=====================================
lib/File/StripNondeterminism.pm
=====================================
@@ -25,7 +25,7 @@ use POSIX qw(tzset);
 
 our($VERSION, $canonical_time, $clamp_time);
 
-$VERSION = '1.1.0'; # <https://semver.org/>
+$VERSION = '1.1.1'; # <https://semver.org/>
 
 sub init() {
 	$ENV{'TZ'} = 'UTC';


=====================================
lib/File/StripNondeterminism/handlers/zip.pm
=====================================
@@ -128,12 +128,12 @@ sub normalize_extra_fields($$) {
 			}
 		} elsif ($id == 0x7875) { # Info-ZIP New Unix Extra Field
 			$result .= substr($field, $pos, 4);
-		  #  Version       1 byte      version of this extra field, currently 1
-		  #  UIDSize       1 byte      Size of UID field
-		  #  UID           Variable    UID for this entry
-		  #  GIDSize       1 byte      Size of GID field
-		  #  GID           Variable    GID for this entry
-		  # (Same format for both central header and local header)
+			#  Version       1 byte      version of this extra field, currently 1
+			#  UIDSize       1 byte      Size of UID field
+			#  UID           Variable    UID for this entry
+			#  GIDSize       1 byte      Size of GID field
+			#  GID           Variable    GID for this entry
+			# (Same format for both central header and local header)
 			if (ord(substr($field, $pos + 4, 1)) == 1) {
 				my $uid_len = ord(substr($field, $pos + 5, 1));
 				my $gid_len = ord(substr($field, $pos + 6 + $uid_len, 1));


=====================================
t/fixtures.t
=====================================
@@ -72,7 +72,7 @@ foreach my $filename (@fixtures) {
 		$normalizer->($in) if defined $normalizer;
 		my @stat_after = lstat $in;
 
-		ok(compare($in, $out) == 0, "Got expected output");
+		ok(compare($in, $out) == 0, "Test output $in matched expected $out");
 
 		# Check that file attributes remain unchanged.
 		foreach my $i (sort keys %STAT) {



View it on GitLab: https://salsa.debian.org/reproducible-builds/strip-nondeterminism/compare/9cad20db37792fbbeb29e981949813322f271ca3...f360b3ce7b86087433a4e1b6d4922f52ce9fcd6c

-- 
View it on GitLab: https://salsa.debian.org/reproducible-builds/strip-nondeterminism/compare/9cad20db37792fbbeb29e981949813322f271ca3...f360b3ce7b86087433a4e1b6d4922f52ce9fcd6c
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.reproducible-builds.org/pipermail/rb-commits/attachments/20190213/8cd730a0/attachment.html>