Ticket #123 (closed enhancement: fixed)

Opened 8 months ago

Last modified 5 months ago

repository generation optimalisation

Reported by: keesj Owned by: keesj
Priority: normal Milestone: 0.2
Component: Build System Version: 0.2
Keywords: Cc:

Description

The gen_repository.sh takes a while to finish. The script calls the reprepro many times at least once for every package.

The provided patch use xargs to give multiple files as argument to reprepro and it "feels" like this make it faster.

Attachments

gen_repo.patch (0.8 kB) - added by vivijim 5 months ago.
using --max-args=1000 to support reprepro 3.x.x

Change History

Changed 8 months ago by vivijim

  • status changed from new to closed
  • resolution set to fixed

Great!

Applied. Thanks.

Changed 7 months ago by rsalveti

  • status changed from closed to reopened
  • resolution fixed deleted

Found a problem :-(

Using reprepro to generate your deb repository: This can take a while: It's coffee time! Error while creating pipe: 24=Too many open files Exporting indices... Error opening file ./dists/mamona/main/binary-arm/Packages.gz.new for writing: Too many open files

I can reproduce this bug only with the 3.x.x series of reprepro. With reprepro 2.2.4 this problem doesn't happen.

For now I'm removing this patch, but it'd be nice to let it working again, it's much faster than the other solution.

Changed 7 months ago by keesj

  • owner set to keesj
  • status changed from reopened to new

Hello, this really sounds like a 3.x.x feature. I will try to find a better solution.

Changed 7 months ago by rsalveti

If you call bug as a feature :-)

Will also test it here and try to find what's happening.

Changed 5 months ago by vivijim

using --max-args=1000 to support reprepro 3.x.x

Changed 5 months ago by vivijim

  • status changed from new to closed
  • resolution set to fixed

Last patch solve all of our problems ;)

Changed 5 months ago by rsalveti

  • status changed from closed to reopened
  • resolution fixed deleted 
(m:omap3430sdp) rsalveti@evamac ~/projects/mamona/build/10-07-08 $ ./tools.git/scripts/gen_repository.sh -s
Starting...
Deploy Directory: /home/rsalveti/projects/mamona/build/10-07-08/tmp/master/omap3430sdp/deploy
Repository Directory: /home/rsalveti/projects/mamona/build/10-07-08/tmp/master/omap3430sdp/deploy/deb-repository
Configuration File: /home/rsalveti/projects/mamona/build/10-07-08/tools.git/conf/gen_repository/mamona/distributions
Sources: yes
Rootfs: no
Image: no

Is this configuration right? [Y/n]
y
Exporting indices...
Deleting files no longer referenced...
Skipping inclusion of 'libstdc++6' '4.1.2-r11' in 'mamona|main|arm', as it has already '4.1.2-r14'.
Exporting indices...
Exporting indices...
Exporting indices...
Error: Too many arguments for command 'include'!
Syntax: reprepro [--delete] include <distribution> <.changes-file>
There have been errors!
xargs: reprepro: exited with status 255; aborting

When creating the repository with sources we still face this problem.

Changed 5 months ago by rsalveti

  • status changed from reopened to closed
  • resolution set to fixed

The problem is that reprepro doesn't get more than one ".changes" in the same command.

Fixed: http://dev.openbossa.org/mamona/gitweb?p=tools.git;a=commitdiff;h=062096e520f3d8d85c580c4c13039118d9b4f4c4;hp=ccf0af98d51ed7ea28b73c9c5389dd707a6651d9

Note: See TracTickets for help on using tickets.