Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » DSDP - Real-Time Software Components (RTSC) » Issue with building large libraries on Windows
Issue with building large libraries on Windows [message #492495] Tue, 20 October 2009 15:47 Go to next message
Laurent Gauthier is currently offline Laurent GauthierFriend
Messages: 2
Registered: October 2009
Junior Member
Hi there,

We are in the process of moving some of our software builds over to RTSC, and are running into an issue that we think is related to very long command lines on Windows.

We create a library with over 400 C source files in it. The target is a C64+ target.

Here is the relevant error message we get, slightly edited to protect the innocent:
... package/lib/./lib/OurLibrary_ReleaseTB/source/chm/src/zzzzzzzzzzzz.o64 package/lib/./lib/OurLibrary_ReleaseTB/source/chm/src/xxxxxx.o64 package/lib/./lib/OurLibrary_ReleaseTB/source/chm/src/yyyyyy.o64 into lib/OurLibrary_ReleaseTB.a64 ..., ...) failed.
make (e=87): The parameter is incorrect.
gmake.exe: *** [lib/OurLibrary_ReleaseTB.a64] Error 87


To work around this problem our previous build system has a work-around built-in that consist in passing to the archiver the list of object files for the libraryin a file rather than on the command line.

Another work-around I have seen implemented in another build system was a creation of the library archive in several pass (creation of the archive with a first set of object files, and addition of more object files in several increments) each of the passes having a limited command line length.

The questions we have are:
1) Is there already a mechanism built-in to avoid this command line limitation (appart from splitting our large library in several smaller libs)? If so a pointer to how to do it would be appreciated.
2) If there is no such mechanism, are there way to add such a mechanism to RTSC as is, or would a change to RTSC be required...


Based on my experience with building large code bases, this is a required feature to support large projects, as splitting in smaller libraries is not always easy/convenient.

Thanks in advance, Laurent.

Re: Issue with building large libraries on Windows [message #492604 is a reply to message #492495] Wed, 21 October 2009 00:04 Go to previous messageGo to next message
Dave Russo is currently offline Dave RussoFriend
Messages: 172
Registered: July 2009
Senior Member
comments below

Laurent Gauthier wrote:
> Hi there,
>
> We are in the process of moving some of our software builds over to
> RTSC, and are running into an issue that we think is related to very
> long command lines on Windows.
>
> We create a library with over 400 C source files in it. The target is a
> C64+ target.
>
[snip]
>
> To work around this problem our previous build system has a work-around
> built-in that consist in passing to the archiver the list of object
> files for the library in a file rather than on the command line.
>
> Another work-around I have seen implemented in another build system was
> a creation of the library archive in several pass (creation of the
> archive with a first set of object files, and addition of more object
> files in several increments) each of the passes having a limited command
> line length.
>
> The questions we have are:
> 1) Is there already a mechanism built-in to avoid this command line
> limitation (apart from splitting our large library in several smaller
> libs)? If so a pointer to how to do it would be appreciated.
> 2) If there is no such mechanism, are there way to add such a mechanism
> to RTSC as is, or would a change to RTSC be required...
>
Since every tool chain has unique options for getting around command
line length limitations, the RTSC build system has to rely on the target
to supply commands that can be executed. For example, the TI archiver
supports a command file approach but the GNU archiver (as far as I know)
does not - it requires the second workaround you describe above.

Since RTSC targets can supply one or more arbitrary commands to perform
the archiving, the problem you are experiencing can be solved in at
least two ways:
1. fix the TI targets to use command files, or
2. create a "target filter" that replaces long lines with command
files

The first option is to fix the ti.targets.ITarget code to always use
command files supported by the TI tool chain. Since ti.targets is
currently bundled with XDCtools, this requires a re-release of XDCtools
from TI and no doubt you don't want to wait for a new release. On the
other hand, this fix should be added to the list of "bugs" to be addressed.

The second option is something that is supported by the RTSC build
engine and allows you to "filter" each command used as part of a build.
In particular, you can write a small filter that changes each archive
command (that would normally be added to the generated makefile) into a
command that uses the -@ option.

Reference docs for target filters are here:
http://rtsc.eclipse.org/cdoc-tip/index.html#xdc/bld/ITargetF ilter.html.

I've created a small example that I believe works with the TI tool chain
and I'll put it in the RTSC SVN repository. The core of the filter is
the following XDCscript file which "implements" the
xdc.bld.ITargetFilter interface:

/*
* ======== archive ========
*/
function archive(container, lib, objList, archArgs, res)
{
/* replace ar command with one that uses the arguments in a file */
res.cmds = mkCommandFile(container, res.cmds);
}

/*
* ======== mkCommandFile ========
*/
function mkCommandFile(archiveName, command)
{
var tokens = command.split(/\s+/);
var cmd = tokens.shift() + " -@" + archiveName + ".cmd";

/* create a command file to hold ar's command line options */
var file = new java.io.BufferedWriter(
new java.io.FileWriter(archiveName + ".cmd"));

/* put archiver options in the file*/
file.write(tokens.shift() + '\n');

/* replace $@ token with actual archive name */
tokens.shift(); /* should be == $@ */
file.write(archiveName + '\n');

/* finally, put all objects to go into the archive */
for (var i = 0; i < tokens.length; i++) {
file.write(tokens[i] + '\n');
}

file.flush();

/* return new ar command to be placed in makefile */
return (cmd);
}
Re: Issue with building large libraries on Windows [message #525641 is a reply to message #492604] Wed, 07 April 2010 09:53 Go to previous messageGo to next message
Patrick Geremia is currently offline Patrick GeremiaFriend
Messages: 79
Registered: July 2009
Member
This is a multi-part message in MIME format.
--------------020206000408070007030009
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

-------- Original Message --------
Subject: Re: Issue with building large libraries on Windows
From: dave russo <d-russo@ti.com>
To:
Date: Wed Oct 21 2009 02:04:37 GMT+0200 (CEST)

> comments below
>
> Laurent Gauthier wrote:
>> Hi there,
>>
>> We are in the process of moving some of our software builds over to
>> RTSC, and are running into an issue that we think is related to very
>> long command lines on Windows.
>>
>> We create a library with over 400 C source files in it. The target is
>> a C64+ target.
>>
> [snip]
>>
>> To work around this problem our previous build system has a
>> work-around built-in that consist in passing to the archiver the
>> list of object files for the library in a file rather than on the
>> command line.
>>
>> Another work-around I have seen implemented in another build system
>> was a creation of the library archive in several pass (creation of the
>> archive with a first set of object files, and addition of more object
>> files in several increments) each of the passes having a limited
>> command line length.
>>
>> The questions we have are:
>> 1) Is there already a mechanism built-in to avoid this command line
>> limitation (apart from splitting our large library in several smaller
>> libs)? If so a pointer to how to do it would be appreciated.
>> 2) If there is no such mechanism, are there way to add such a
>> mechanism to RTSC as is, or would a change to RTSC be required...
>>
> Since every tool chain has unique options for getting around command
> line length limitations, the RTSC build system has to rely on the target
> to supply commands that can be executed. For example, the TI archiver
> supports a command file approach but the GNU archiver (as far as I know)
> does not - it requires the second workaround you describe above.
>
> Since RTSC targets can supply one or more arbitrary commands to perform
> the archiving, the problem you are experiencing can be solved in at
> least two ways:
> 1. fix the TI targets to use command files, or
> 2. create a "target filter" that replaces long lines with command
> files
>
> The first option is to fix the ti.targets.ITarget code to always use
> command files supported by the TI tool chain. Since ti.targets is
> currently bundled with XDCtools, this requires a re-release of XDCtools
> from TI and no doubt you don't want to wait for a new release. On the
> other hand, this fix should be added to the list of "bugs" to be addressed.
>
> The second option is something that is supported by the RTSC build
> engine and allows you to "filter" each command used as part of a build.
> In particular, you can write a small filter that changes each archive
> command (that would normally be added to the generated makefile) into a
> command that uses the -@ option.
>
> Reference docs for target filters are here:
> http://rtsc.eclipse.org/cdoc-tip/index.html#xdc/bld/ITargetF ilter.html
>
> I've created a small example that I believe works with the TI tool chain
> and I'll put it in the RTSC SVN repository. The core of the filter is
> the following XDCscript file which "implements" the
> xdc.bld.ITargetFilter interface:
>
> /*
> * ======== archive ========
> */
> function archive(container, lib, objList, archArgs, res)
> {
> /* replace ar command with one that uses the arguments in a file */
> res.cmds = mkCommandFile(container, res.cmds);
> }
>
> /*
> * ======== mkCommandFile ========
> */
> function mkCommandFile(archiveName, command)
> {
> var tokens = command.split(/\s+/);
> var cmd = tokens.shift() + " -@" + archiveName + ".cmd";
>
> /* create a command file to hold ar's command line options */
> var file = new java.io.BufferedWriter(
> new java.io.FileWriter(archiveName + ".cmd"));
>
> /* put archiver options in the file*/
> file.write(tokens.shift() + '\n');
>
> /* replace $@ token with actual archive name */
> tokens.shift(); /* should be == $@ */
> file.write(archiveName + '\n');
>
> /* finally, put all objects to go into the archive */
> for (var i = 0; i < tokens.length; i++) {
> file.write(tokens[i] + '\n');
> }
>
> file.flush();
>
> /* return new ar command to be placed in makefile */
> return (cmd);
> }
>
>
Dave,

could you be a little bit more specific on how to implement 2) in
practice? I have hard-time understanding how to fit archive function
above in my config.bld (see attachment).

--
Patrick Geremia
Texas Instruments (http://www.ti.com)
Phone: +33 4 93 22 26 33
Email: p-geremia@ti.com
Availability: http://meetwith.me/patrickgeremia

--------------020206000408070007030009
Content-Type: text/plain;
name="config.bld"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="config.bld"

dmFyIGJ1aWxkID0geGRjLnVzZU1vZHVsZSgneGRjLmJsZC5CdWlsZEVudmly b25tZW50Jyk7
CnByaW50IChidWlsZC5ob3N0T1NOYW1lKTsKdmFyIHRpQ2d0RGlyID0gamF2 YS5sYW5nLlN5
c3RlbS5nZXRlbnYoIlRJX0NHVF9ESVIiKTsKdmFyIHRpQ3NsRGlyID0gamF2 YS5sYW5nLlN5
c3RlbS5nZXRlbnYoIlRJX0NTTF9ESVIiKTsKdmFyIGhvc3RDc2xEaXIgPSBq YXZhLmxhbmcu
U3lzdGVtLmdldGVudigiVElfSE9TVF9DU0xfRElSIik7CnZhciBob3N0SW50 cmluc2ljc0Rp
ciA9IGphdmEubGFuZy5TeXN0ZW0uZ2V0ZW52KCJUSV9IT1NUX0lOVFJJTlNJ Q1NfRElSIik7
CnZhciBnbnVDZ3REaXIgPSBqYXZhLmxhbmcuU3lzdGVtLmdldGVudigiR05V X0NHVF9ESVIi
KQoKdmFyIHBrZ05hbWUgPSBQa2cubmFtZS5yZXBsYWNlKC9cLi9nLCAiXyIp OwoKLyoKICog
ID09PT09PT09IGFyY2hpdmUgPT09PT09PT0KICovCmZ1bmN0aW9uIGFyY2hp dmUoY29udGFp
bmVyLCBsaWIsIG9iakxpc3QsIGFyY2hBcmdzLCByZXMpCnsKICAgIC8qIHJl cGxhY2UgYXIg
Y29tbWFuZCB3aXRoIG9uZSB0aGF0IHVzZXMgdGhlIGFyZ3VtZW50cyBpbiBh IGZpbGUgKi8K
ICAgIHJlcy5jbWRzID0gbWtDb21tYW5kRmlsZShjb250YWluZXIsIHJlcy5j bWRzKTsKfQoK
LyoKICogID09PT09PT09IG1rQ29tbWFuZEZpbGUgPT09PT09PT0KICovCmZ1 bmN0aW9uIG1r
Q29tbWFuZEZpbGUoYXJjaGl2ZU5hbWUsIGNvbW1hbmQpCnsKICAgIHZhciB0 b2tlbnMgPSBj
b21tYW5kLnNwbGl0KC9ccysvKTsKICAgIHZhciBjbWQgPSB0b2tlbnMuc2hp ZnQoKSArICIg
LUAiICsgYXJjaGl2ZU5hbWUgKyAiLmNtZCI7CgogICAgLyogY3JlYXRlIGEg Y29tbWFuZCBm
aWxlIHRvIGhvbGQgYXIncyBjb21tYW5kIGxpbmUgb3B0aW9ucyAqLwogICAg dmFyIGZpbGUg
PSBuZXcgamF2YS5pby5CdWZmZXJlZFdyaXRlcigKICAgICAgICBuZXcgamF2 YS5pby5GaWxl
V3JpdGVyKGFyY2hpdmVOYW1lICsgIi5jbWQiKSk7CgogICAgLyogcHV0IGFy Y2hpdmVyIG9w
dGlvbnMgaW4gdGhlIGZpbGUqLwogICAgZmlsZS53cml0ZSh0b2tlbnMuc2hp ZnQoKSArICdc
bicpOwoKICAgIC8qIHJlcGxhY2UgJEAgdG9rZW4gd2l0aCBhY3R1YWwgYXJj aGl2ZSBuYW1l
ICovCiAgICB0b2tlbnMuc2hpZnQoKTsgLyogc2hvdWxkIGJlID09ICRAICov CiAgICBmaWxl
LndyaXRlKGFyY2hpdmVOYW1lICsgJ1xuJyk7CgogICAgLyogZmluYWxseSwg cHV0IGFsbCBv
YmplY3RzIHRvIGdvIGludG8gdGhlIGFyY2hpdmUgKi8KICAgIGZvciAodmFy IGkgPSAwOyBp
IDwgdG9rZW5zLmxlbmd0aDsgaSsrKSB7CiAgICAgICAgZmlsZS53cml0ZSh0 b2tlbnNbaV0g
KyAnXG4nKTsKICAgIH0KCiAgICBmaWxlLmZsdXNoKCk7CgogICAgLyogcmV0 dXJuIG5ldyBh
ciBjb21tYW5kIHRvIGJlIHBsYWNlZCBpbiBtYWtlZmlsZSAqLwogICAgcmV0 dXJuIChjbWQp
Owp9CgppZiAodGlDZ3REaXIgJiYgdGlDc2xEaXIpCnsKCXZhciB0aUNjb3B0 cyA9ICItLXZl
cmJvc2VfZGlhZ25vc3RpY3MgLS13cml0ZV9kaWFnbm9zdGljc19maWxlIC0t ZGlzcGxheV9l
cnJvcl9udW1iZXIgLS1kaWFnX2Vycm9yPTIyNSAtayAtLWRpYWdfZXJyb3I9 OSAtLWRpYWdf
d2FybmluZz0xNzkgLS1kaWFnX3dhcm5pbmc9ODgwIC0tZGlhZ19yZW1hcms9 MTg4ICI7Cgl2
YXIgdGlDc2xJbmNQYXRoID0gIi1JIiArIHRpQ3NsRGlyICsgImluYyAiOwoJ dmFyIHRpSW5j
cyA9IHRpQ3NsSW5jUGF0aDsKCXZhciB0aURlZnMgPSAiLURzd3Bmb3JtX3Rh cmdldF9fPXRp
L3diaS9jb21tb24vc3dwZm9ybS9jNnguaCAiOwoJdmFyIHRpNjRQID0geGRj Lm1vZHVsZSgi
dGkudGFyZ2V0cy5DNjRQIik7Cgl2YXIgdGk2NFBlID0geGRjLm1vZHVsZSgi dGkudGFyZ2V0
cy5DNjRQX2JpZ19lbmRpYW4iKTsKCXRpNjRQLmNjT3B0cy5wcmVmaXggPSB0 aUNjb3B0czsK
CXRpNjRQZS5jY09wdHMucHJlZml4ID0gdGlDY29wdHM7Cgl0aTY0UC5wcm9m aWxlc1twa2dO
YW1lICsgIl9hc3NlcnRfb25fZGVidWciXT0geyAKCQkJY29tcGlsZU9wdHM6 IAoJCQl7IAoJ
CQkJY29wdHM6ICItZyAiLAoJCQkJaW5jczogdGlJbmNzLAoJCQkJZGVmczog dGlEZWZzICsg
Ii1EUkFDQ01fQVNTRVJUPTEiCgkJCX0gCgkJfTsKCXRpNjRQLnByb2ZpbGVz W3BrZ05hbWUg
KyAiX2Fzc2VydF9vbl9yZWxlYXNlIl09IHsgCgkJCWNvbXBpbGVPcHRzOiAK CQkJeyAKCQkJ
CWNvcHRzOiAiLS1vcHRfbGV2ZWw9MyAtLWdlbl9vcHRfaW5mbz0yICIsCgkJ CQlpbmNzOiB0
aUluY3MsCgkJCQlkZWZzOiB0aURlZnMgKyAiLURSQUNDTV9BU1NFUlQ9MSIK CQkJfSAKCQl9
OwoJdGk2NFAucHJvZmlsZXNbcGtnTmFtZSArICJfYXNzZXJ0X29mZl9kZWJ1 ZyJdPSB7IAoJ
CQljb21waWxlT3B0czogCgkJCXsgCgkJCQljb3B0czogIi1nICIsCgkJCQlp bmNzOiB0aUlu
Y3MsCgkJCQlkZWZzOiB0aURlZnMKCQkJfSAKCQl9OwoJdGk2NFAucHJvZmls ZXNbcGtnTmFt
ZSArICJfYXNzZXJ0X29mZl9yZWxlYXNlIl09CgkJeyAKCQkJY29tcGlsZU9w dHM6IAoJCQl7
IAoJCQkJY29wdHM6ICItLW9wdF9sZXZlbD0zIC0tZ2VuX29wdF9pbmZvPTIg IiwKCQkJCWlu
Y3M6IHRpSW5jcywKCQkJCWRlZnM6IHRpRGVmcwoJCQl9IAoJCX07Cgl0aTY0 UGUucHJvZmls
ZXNbcGtnTmFtZSArICJfYXNzZXJ0X29uX2RlYnVnIl0gPSB0aTY0UC5wcm9m aWxlc1twa2dO
YW1lICsgIl9hc3NlcnRfb25fZGVidWciXTsKCXRpNjRQZS5wcm9maWxlc1tw a2dOYW1lICsg
Il9hc3NlcnRfb25fcmVsZWFzZSJdID0gdGk2NFAucHJvZmlsZXNbcGtnTmFt ZSArICJfYXNz
ZXJ0X29uX3JlbGVhc2UiXTsKCXRpNjRQZS5wcm9maWxlc1twa2dOYW1lICsg Il9hc3NlcnRf
b2ZmX2RlYnVnIl0gPSB0aTY0UC5wcm9maWxlc1twa2dOYW1lICsgIl9hc3Nl cnRfb2ZmX2Rl
YnVnIl07Cgl0aTY0UGUucHJvZmlsZXNbcGtnTmFtZSArICJfYXNzZXJ0X29m Zl9yZWxlYXNl
Il0gPSB0aTY0UC5wcm9maWxlc1twa2dOYW1lICsgIl9hc3NlcnRfb2ZmX3Jl bGVhc2UiXTsK
CXRpNjRQLnJvb3REaXIgPSB0aUNndERpcjsKCXRpNjRQZS5yb290RGlyID0g dGlDZ3REaXI7
CglidWlsZC50YXJnZXRzID0gW3RpNjRQLCB0aTY0UGVdOwp9CmVsc2UKewoJ cHJpbnQoIkVu
dmlyb25tZW50LiB2YXJpYWJsZXMgVElfQ0dUX0RJUiBhbmQgVElfQ1NMX0RJ UiBub3Qgc2V0
IC4uLiIpOwoJcHJpbnQoIlNraXBwaW5nIFRJIDY0UCB0YXJnZXQgLi4uIik7 CglwcmludCgi
U2tpcHBpbmcgVEkgNjRQX2JpZ19lbmRpYW4gdGFyZ2V0IC4uLiIpOwp9Cgpp ZiAoaG9zdENz
bERpciAmJiBob3N0SW50cmluc2ljc0RpciAmJiBnbnVDZ3REaXIpCnsKCXZh ciBnbnVDY29w
dHMgPSAiLXN0ZD1jOTkgLW0zMiAtcGVkYW50aWMgLVdhbGwgLVdlcnJvci1p bXBsaWNpdC1m
dW5jdGlvbi1kZWNsYXJhdGlvbiAtRFNJTVVMQVRJT04gLURMSVRUTEVfRU5E SUFOX0hPU1Qg
LURUTVMzMjBDNjRQWCAtRFVTRV9TSU1VTEFURURfUkFDICI7Cgl2YXIgZ251 SW5jcyA9ICIt
SSIgKyBob3N0Q3NsRGlyICsgImluY2x1ZGUgIiArICItSSIgKyBob3N0SW50 cmluc2ljc0Rp
ciArICIgIjsKCXZhciBnbnVEZWZzID0gIi1EU0lNVUxBVElPTj0xIC1ETElU VExFX0VORElB
Tl9IT1NUPTEgLURUTVMzMjBDNjRQWD0xIC1Ec3dwZm9ybV90YXJnZXRfXz10 aS93YmkvY29t
bW9uL3N3cGZvcm0vYzZ4LmgiOwoJaWYgKGJ1aWxkLmhvc3RPU05hbWUgPT0g IkxpbnV4IikK
CXsKCQl2YXIgZ251ID0geGRjLnVzZU1vZHVsZSgnZ251LnRhcmdldHMuTGlu dXg4NicpOyAK
CX0KCWVsc2UKCXsKCQl2YXIgZ251ID0geGRjLnVzZU1vZHVsZSgnZ251LnRh cmdldHMuTWlu
Z3cnKTsgCgl9CglnbnUuY2NPcHRzLnByZWZpeCA9IGdudUNjb3B0czsKCgln bnUucHJvZmls
ZXNbcGtnTmFtZSArICJfYXNzZXJ0X29uX2RlYnVnIl09IHsgCgkJCWNvbXBp bGVPcHRzOiAK
CQkJeyAKCQkJCWNvcHRzOiBnbnUucHJvZmlsZXNbImRlYnVnIl0uY29tcGls ZU9wdHMuY29w
dHMsCgkJCQlpbmNzOiBnbnVJbmNzLAoJCQkJZGVmczogIi1EUkFDQ01fQVNT RVJUPTEgIiAr
IGdudURlZnMKCQkJfSAKCQl9OwoJZ251LnByb2ZpbGVzW3BrZ05hbWUgKyAi X2Fzc2VydF9v
bl9yZWxlYXNlIl09IHsgCgkJCWNvbXBpbGVPcHRzOiAKCQkJeyAKCQkJCWNv cHRzOiBnbnUu
cHJvZmlsZXNbInJlbGVhc2UiXS5jb21waWxlT3B0cy5jb3B0cywKCQkJCWlu Y3M6IGdudUlu
Y3MsCgkJCQlkZWZzOiAiLURSQUNDTV9BU1NFUlQ9MSAiICsgZ251RGVmcwoJ CQl9IAoJCX07
CglnbnUucHJvZmlsZXNbcGtnTmFtZSArICJfYXNzZXJ0X29mZl9kZWJ1ZyJd PSB7IAoJCQlj
b21waWxlT3B0czogCgkJCXsgCgkJCQljb3B0czogZ251LnByb2ZpbGVzWyJk ZWJ1ZyJdLmNv
bXBpbGVPcHRzLmNvcHRzLAoJCQkJaW5jczogZ251SW5jcywKCQkJCWRlZnM6 IGdudURlZnMK
CQkJfSAKCQl9OwoJZ251LnByb2ZpbGVzW3BrZ05hbWUgKyAiX2Fzc2VydF9v ZmZfcmVsZWFz
ZSJdPQoJCXsgCgkJCWNvbXBpbGVPcHRzOiAKCQkJeyAKCQkJCWNvcHRzOiBn bnUucHJvZmls
ZXNbInJlbGVhc2UiXS5jb21waWxlT3B0cy5jb3B0cywKCQkJCWluY3M6IGdu dUluY3MsCgkJ
CQlkZWZzOiBnbnVEZWZzCgkJCX0gCgkJfTsKCglnbnUucm9vdERpciA9IGdu dUNndERpcjsK
CWJ1aWxkLnRhcmdldHMuJGFkZChnbnUpOwp9CmVsc2UKewoJcHJpbnQoIkVu dmlyb25tZW50
IHZhcmlhYmxlcyBUSV9IT1NUX0NTTF9ESVIsIFRJX0hPU1RfSU5UUklOU0lD U19ESVIgYW5k
IEdOVV9DR1RfRElSIG5vdCBzZXQgLi4uIik7CglwcmludCgiU2tpcHBpbmcg R05VIHRhcmdl
dHMgLi4uIik7Cn0KCg==
--------------020206000408070007030009--
Re: Issue with building large libraries on Windows [message #525732 is a reply to message #525641] Wed, 07 April 2010 15:32 Go to previous messageGo to next message
Dave Russo is currently offline Dave RussoFriend
Messages: 172
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------010608060505010600090503
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Target filters must be modules that implement the ITargetFilter
interface. The functions shown below (archive and mkCommandFile) must
be in the .xs file of the module that implements ITargetFilter.

See the attached example package basic.filter.

On 4/7/2010 2:53 AM, Patrick Geremia wrote:
> -------- Original Message --------
> Subject: Re: Issue with building large libraries on Windows
> From: dave russo <d-russo@ti.com>
> To:
> Date: Wed Oct 21 2009 02:04:37 GMT+0200 (CEST)
>
>> comments below
>>
>> Laurent Gauthier wrote:
>>> Hi there,
>>>
>>> We are in the process of moving some of our software builds over to
>>> RTSC, and are running into an issue that we think is related to very
>>> long command lines on Windows.
>>>
>>> We create a library with over 400 C source files in it. The target is
>>> a C64+ target.
>>>
>> [snip]
>>>
>>> To work around this problem our previous build system has a
>>> work-around built-in that consist in passing to the archiver the list
>>> of object files for the library in a file rather than on the command
>>> line.
>>>
>>> Another work-around I have seen implemented in another build system
>>> was a creation of the library archive in several pass (creation of
>>> the archive with a first set of object files, and addition of more
>>> object files in several increments) each of the passes having a
>>> limited command line length.
>>>
>>> The questions we have are:
>>> 1) Is there already a mechanism built-in to avoid this command line
>>> limitation (apart from splitting our large library in several smaller
>>> libs)? If so a pointer to how to do it would be appreciated.
>>> 2) If there is no such mechanism, are there way to add such a
>>> mechanism to RTSC as is, or would a change to RTSC be required...
>>>
>> Since every tool chain has unique options for getting around command
>> line length limitations, the RTSC build system has to rely on the
>> target to supply commands that can be executed. For example, the TI
>> archiver supports a command file approach but the GNU archiver (as far
>> as I know) does not - it requires the second workaround you describe
>> above.
>>
>> Since RTSC targets can supply one or more arbitrary commands to
>> perform the archiving, the problem you are experiencing can be solved
>> in at least two ways:
>> 1. fix the TI targets to use command files, or
>> 2. create a "target filter" that replaces long lines with command
>> files
>>
>> The first option is to fix the ti.targets.ITarget code to always use
>> command files supported by the TI tool chain. Since ti.targets is
>> currently bundled with XDCtools, this requires a re-release of
>> XDCtools from TI and no doubt you don't want to wait for a new
>> release. On the other hand, this fix should be added to the list of
>> "bugs" to be addressed.
>>
>> The second option is something that is supported by the RTSC build
>> engine and allows you to "filter" each command used as part of a
>> build. In particular, you can write a small filter that changes each
>> archive command (that would normally be added to the generated
>> makefile) into a command that uses the -@ option.
>>
>> Reference docs for target filters are here:
>> http://rtsc.eclipse.org/cdoc-tip/index.html#xdc/bld/ITargetF ilter.html
>>
>> I've created a small example that I believe works with the TI tool
>> chain and I'll put it in the RTSC SVN repository. The core of the
>> filter is the following XDCscript file which "implements" the
>> xdc.bld.ITargetFilter interface:
>>
>> /*
>> * ======== archive ========
>> */
>> function archive(container, lib, objList, archArgs, res)
>> {
>> /* replace ar command with one that uses the arguments in a file */
>> res.cmds = mkCommandFile(container, res.cmds);
>> }
>>
>> /*
>> * ======== mkCommandFile ========
>> */
>> function mkCommandFile(archiveName, command)
>> {
>> var tokens = command.split(/\s+/);
>> var cmd = tokens.shift() + " -@" + archiveName + ".cmd";
>>
>> /* create a command file to hold ar's command line options */
>> var file = new java.io.BufferedWriter(
>> new java.io.FileWriter(archiveName + ".cmd"));
>>
>> /* put archiver options in the file*/
>> file.write(tokens.shift() + '\n');
>>
>> /* replace $@ token with actual archive name */
>> tokens.shift(); /* should be == $@ */
>> file.write(archiveName + '\n');
>>
>> /* finally, put all objects to go into the archive */
>> for (var i = 0; i < tokens.length; i++) {
>> file.write(tokens[i] + '\n');
>> }
>>
>> file.flush();
>>
>> /* return new ar command to be placed in makefile */
>> return (cmd);
>> }
>>
>>
> Dave,
>
> could you be a little bit more specific on how to implement 2) in
> practice? I have hard-time understanding how to fit archive function
> above in my config.bld (see attachment).
>

--------------010608060505010600090503
Content-Type: application/gzip;
name="filter.tar.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="filter.tar.gz"

H4sIANqkvEsAA+09a1fjOLL9lfwKTZqzOBCcd7IHtvcODXQPZ5qGA/Td2UMY 2kmUxI1jZ22H
x8zy329VSXLkR4CeTad397rOTCexpZJUqpdKJdGzArtfGdpOyP3Kq28DVYBO q/WqWqu16+0q
/a41m/Qp4VWt1mh1avV6vVaDcvVms/6Ktb5Rf2IwC0LLZ+zVwH+6XDDl/dD3 3FX0aYXQ0+f/
sD/2zPtBf8lt4AQ3cb4XzH+zg/Nf79TatU61Ae9rzWod5r+65H5kwv/z+a9s /lBgm4y9kcAU
D0RP6PUeAyqNeMgEp7BwbIUssCdT54FNfdsNA2axCQ8Ca8ShQqUw4aHlufB2 4g1mDie0zHbH
3LehLOA3e87APLogrO8IaeH3gu3CbLh9vlNgANQ3BL1/shGte7LEqc+H9j3z hoxb/XFUbmAH
U8d64ANZsEKffc8d2iN2HkLfR3OcrLgBKP+6UdzNKOVaEyoCLx8L33velgUZ 8h8su41n5L/e
ajZQ/zearXa72sZyjU6tkcv/KqCyGRd/y++P7VtdvEBihjO3H9qeq94aIBih ZbvcLzPH7pWZ
1/vywQ7CMhXY80dBmfk8KIFEoxiRgjCKCrVUIaZpFku7egEpcgLzDiuyrfnP dElomCSaiZLw
00QJTReE9ngwLyj7CuWkkmHWdMrdASiwqReE21Pf64M+QIkPQj5VGgPGY/Yn g4BtgQrouuvG
8fn7EiEMx3ZgKhUif2JPhMIB2IJiavDJJnoPcxzrQlea61gblUwhOTt9bzKF 0SyYHflWnx2h
tcts5FkOaFlepkKLZkihX9IMSZOhCC9+Lpgk7KA2m6q/6YKBN/P7XBTUKKyG ZQZ+X1oC7SFo
OZ4odD4bQiGNC6Y+31Z8gF8TbBB6wq4oGiXZArjia3ki0cz6XxZzAzzc6Lob 8KEazOQPx3Zv
FjAHvtI5oz8cvQWqlKEX3kiTYCy3iD8I/ZKYA9rXxVd2J10Ou6eVw58LWAiG EGglE3L+r8qv
azkPv+E0xXpB7+7scPyEFH9vFf8kxOz/1OrfABHQNVtmG8/5/7U6rf+atUa1 1mk00P9v1aq5
/V8FoPkbDIBzecClpQRNh4zM+L0F/j3fCKTODVCfnN6MTA9K+++o7Bt2WXQ9 dOd97hSvdv+9
mT2HFGTK/5JDAM/Lfyfy/xv1Jsl/s5PL/ypATjkjPjClbb+slavl6pU0/toK fld/sCdc2tiz
Y+vm39/m5TCHmPyj4HP31pxYN8tsA2W8vVj+UfTJ/nc6tVa12hH2v5bL/yrg deH6l4P9t5+O
Phzsn3z6eAEWvVOwhy7/BzPK68bQdgeBCH9NrXAMT64/nR9CjcOP/3tdKpUK 3i33fXvAGTw7
3bv4Cer/WjHNytj+Apol2KUf9mTq+WGwWxn0KqHnObD8qsB63bf8B/FbWZ4g hu/s5AT7g7X8
0Oe8QuFBx6ngD6jjg87ph0EFuBa/q8/tSa2lXlY+2O7sHt9QO9eN63r1ulq9 rrWuuTuKtSZo
8O49tAhdhv9E9A+d4QIsC+1hNlksXCs9SZa9s/fngDT27AIeHl7QY4H7NWBH 5NVyrVQ4/fn9
Emi5++0IN2+DSFX46eT84gRHQ7XkkL43a+fwAojp/75jczdc+jbg1+z/NepV iv/Wa/n+3yog
a/6X7QY8b//rNP/VVrMOn1Cu1Wk1c/u/Csiw/7Xc/ue2/8W2Pzf//9GQpf+X HQZ6Rv9X641k
/ke7mq//VgPJHSQ9DmQKfohnghyKqDCbBRxzLcJxPHZkUqJHLFmE9qGy4kwS fzzMdOwN8gDS
6iBL/ufGbzltPOf/tRvVyP+rduro/7Vz/281cAuDJysPEviGgddvwxgnwAWX xYuTkw/nuKeD
ZW5loewyuA1auQW77/lBUVTwxF5ssgJuFY29IDwhzFSSg8YQGWBQWNcFH60J 32HFlHoplsXW
r+Vbk2BHVnksPAI+QihzLV6MUwayAe2j7NJ+u3kKFbGzoppRDG1TaLXAxLe4 942fpu954YGN
zUgSISlCu9Jv31c6ZtWsvm1UaDs6kLjJR/pzO45+Y+TOIvyyxAY0Ib9qrcwb GfChNXPCyqjf
/3P7E5C98HZmOwOFBvfmsIdI5kpFIQIXj3b5Los+d7gV8OKVJENgOuDdheOt rZcVv6xesTdx
Wu8Whp4v0iQMYizoCbNdFutXSc5HvKhsCEtjuahhLF2pqJ14+TTqQuJ3NIKM nv0RFL8X1p7m
HNzuALZZi7Pj2hqsnDlilXUoUZKrHBgzoNQTnENcYhdF9cfC2uOu4uTCyuzf U/7fsgzAc/5f
u92Y7/9X6+T/VfP9/5VABVy6vs+tkDOLBRxM/4BJURfunR0w5bvJbB5c+VJq L/p+JKTkGCKB
YB07wCQmfBOlBUT45O5iQeURWIPBmXhliCSCYlnqBn6PC909x9lhoT/jZZnB 45z3fXsaKilU
bDtPQCg8YroPahZSKjbIcHUXPv4SV0FSyOHN1pZSR/RPpLTexCtc2le7UQFM PHzD5AA+iOU3
5kb1pPwURdpdCV+f9L4A2wQGlhZqJCjNMfF7Psd0eM/7s9DqobEBJ1hiKUt1 6FghDGsClR+X
Ov9Z8g+tm8tMAHh2/VdtCP+v1u4I/6/ZrrZy+V8FvLbdvjMbcFaUUl6xXWAF 13KIDchdK6YT
DSegB+KJhkewUIQa1/jGwB/Atv2yyp2HH7eXVyqd0OfhzHeZUS3lS73vDFny j6psmW183fkv
kv9mu5Pv/6wCFun/ZaaAPaf/q+2WFv/D838tfJ3r/xXAPOomdTOof9Luhh3p 8P4YKLQpVXiu
sf+rYKH/NxwtrY3n5L9Wa8n8zw5IPsX/avVc/lcCuAyB+ZbRqFnAj2W8KyNQ b+KyBBTAqe+N
fGtiCicQq9PXPPn7PxBi8o/hrO9x/rtdVfnfrXq9Ic5/t3P5XwWkzn8rHnjB +e8Rd7lvhRzP
fkcxThX6zDwFjsiXcwpcC7Cmj4LTU2/IvFk4nYVRYTzZImJdT50Fj2FW46JA 7X+j65Mh/ys+
/w3GvtqW8Z8OLPzE+e9qLv8rAbL/XGzZnd6MErtSSkJPRWxo33ND8AMC3Jmi GmeHUMHld+yM
jw7vpwbJU/HyV3a1pXYRrnHbA0Oc6rik6eH7zaLp86lj9blR6ZqVUZkVu2ax hIhTsSbqzDrq
knXbtcMFB1yTxQwVbTpGOcZAduaRWaVsnsWfLhi1gKQgBSOI8cW6tUzbM/GI nCFOl2pKRcZ/
8edlccAdHvLilaGOqYojpFPfvgU1ZYICS+B8OxsOuc8HfwMNyn0j2tZKtivf YyulbNzzbcLf
5cZT/P3A9tVLdUT6CAZtW479G6cI/5hbA7AF0gwAjUwgGMtAJQq+l+YCnc2h 5QTZR9y/xQUE
0HWfTzzAOor68MvBPuthjB/rgmiLCUyf6lZfI3Y9OwReVUefZbtQTn5LF4s4 RExmaoJlCXvI
jPg7GNgBH7If3tAeCO5TrGUWEO93C2uFNUBn3uHMG8XXXfc1OweL/eHo7cHR mTjVideh+DBY
z39gd2CFORPpgzbacPglrOMAqnZd7LuOUOL5H7CJhbU1pp/uNoNZLwh9o1rW njlWEB65A35/
MjQ2KhulEgp/1xWIxT5nRBo6/U8n6EUrpWIGerDOxhPoJR21HpvR4OaXCohO qC5cAz+c4aYM
kZ54ydSvTNAYS/FBaaUXM3w15yIbKZygdsL+GDixpDa55sH32BQkyZDqXFT5 D16z8Bz5vtW9
BUA+7gYz4Gwe7a4JTkcChuzaGoKWvNakQJIxJaWSp6KOIB/tsDmHzXkqk6J0 YYAiqzaaF7IX
VANJDxaQSL41tFErbonkHcdGAt/jfQtz9+7gneduhGxs3eLeb4+H6NiPPe8G dcWN690pfxo0
hasc/7myJwdbbG8yO0Bk3Iz51UiDAzAiRtqulNMEnt/TIPaG3JnjZBMjou0C csxpL/oc52ec
C6DDP2AakNyaBRf6WUY84ho6mledtKE3RRqgbRciaHwBZ5J54COUdHFMYkxa Q03BKzF7ropS
+ao8DvknKmMIUmrCLU/4W+6DpvynQnalUXDQhMFQdGaJ1krRdAIWmE5DEDHZ Q5zVUpxQMHZg
MmQvZDPLAWyDh7LGmiJ/CT/Rm5IspTKdniCg2pSXF8rQRRtXwAyMeAbt5Atr sNqchEnTqXqX
uLkmKo+Al39I3e4zmWcvZMPx+vQly5LqnSCHWA0ZTeu8NfU0ZTV1GuPMklmD 6YLGYZbx/pq+
wy2X+TOx5JUGhFI5qKwkbXLEkYDptjJ6Go0kUdXCJI1s66qVoh7t7MBAwu11 4+y4lFElJWA0
AEzmoHJ0y4qQ3Zkd2OFcWWvtACYNr7oyBuvEuyadu5hjV+mudw1J9W6JliTr hjYhqQkrFbVc
Dl9oJIEymDqwPMAbe5S3n5GSQjWyUlF0lOiBYrkoA0UJBT41bekA4dVDPx8c nkKPUJ1s12Js
+s9/smTpd0e/6MX1loWZd0Pb1TXMY5agdEMkj+pfnMKPGVOj+2hDZxaMjcUa XqizJ1S8pu80
JY7HWSxglDcynmOksk/xdfGqNJ/23a31iu6r63LBLlBqSLvfWYHmhMkkJxz/ sbgFSBpIkvnY
1VQHUMEopWWiK7SD8NLV8SmQ1wdv5qucq40AxHfqAbuD1l4kgqqu1B+KBAnF kRT3Y6vve5ry
xZZwaQnmKxqeYOoed7y7RY3zyTR8YG8y3wUwCuDgdYMKlZj6smgU++/e4xJD DANjBuDsRSow
u8rRx308eLR9ZML/6wYuE6bQDq0Y2G4ZHpbVL/jE/pTK3W70dd2QxCsBZLYB ckVdmtyAicss
cXaMQjph28MF5DZPfzr5+PcdUtBCNet+Y8JzTKjW3STOJ+VGmOh46Pad50eZ ejJJz+aDMpkO
jMIucgpo/T+2gnFC9KQbMKRFC5l9tXMsTDVPLc2EjkE1lH7d7W5k6CCU5AGl O1ONSFQvu5Vu
9+ryV/rYjMRW14tY7Qe8pTOp07Crl/AP2n34SKq2x0WK6IDiIDpF3+sGijwl axZ6sNiy+zBl
D1E+JUy37lG5HG96s/wHQjKU0yJWcptguDc1kdPNdtyJH8xEQhUhIW4y07qR 3G7haGv6MaZ7
SAdoPY16p6LmIDTbWsPAh5oCwLTRfVkZozK+9YDOW3LIhsguo0v04uhKtIUQ RexdDkoHBt2L
YhDxtYRkCNSU83iVfLjnI6NcXiUsLTKC7QoOTbKWJXhA2mlgq263crVV0ViJ MkEnU3nfq3qa
ZcYH1iIbHrGkBdZRFmJ/ZVX2pz8xekaMaqY4FUG0Df+aYIvBlxRIKO9f5/gE 5wN9LqEOIU4Z
dU0QonIxFzhRCOh6KT5k369kj+bhmGqZHsihbUObaWyPhexfSuwifkqLlWTP lAhwMLQTWECD
r9sDGVGsEnHTR49qWWGqKi79XS9UTCaWtliIVnHgy0/Qg7aDCJXwXIsmE54A /IfVaaEHjO9N
Q3ti/0au/g6M4OMngSjgfEIjuOF8GuHSO+JNoWHV+gNdQWoRv1qseAcaoW/5 gyLmQft4O6Tn
mhGWc5Apzv5mu406LLq56BAInnfHKJos11EWqc6KHu+j8xVzmbNFL8rwDL/j 4WJoqpdYNMTp
R064XGBEiBTZZi7MWzCPLET+xEIlFikwwpTy5uWhayoEpvp47+fD/eOD9yd7 H851a50plTrj
ZnrXwr5IPte9a+DF4Aa8CDyzHeu1Y8PUFk2zUlR91URPBtsqv152TVInYJ5K +qJUl4LYGOW5
cjXr5PwMhNejjzKjLv4EV2bMQe3SCuDorBSr/mRtOqYdLxGTyXTRhZ7I997F yuGPQmz/Vx6E
+xb3/z2R/1VvNJqJ/M9mq5Gf/14JpPI/NB54QQoIrBdQMXqU4k8V/cgcOGCh 8WgQ1VbraKwJ
3/pgPHt0iHwgbMrFEZ1s7I8t25XuL/n8VBmwPohGJO5AIBd9AAuJgbqh701Y 6D+gYZn7Dmim
AIPaVaS2xjN34HNAAmaSzi+ZWb288/ybABfCoieElmKlLo8NEeyvP0G/nzDw iR1q8QFJMmrW
c7nqsLSLmDgzDynIrBmmn7PSo7MqhUWOJer0j5bx6Xzv/WFJDMGjo/nhfCRl jCiwCbosItUF
90MQsTpjqc7x4xyIQClhor6iw0SzBK4JBSY+z0+Hf2YBnccC5+h8NgXvARaE dKpTXBFQli4+
oMIOCBdfI7BaAX1OHK39rAJujL19wGWqJIRcNqGng09g4S6aVwtWvLsbRqJ3 sBxrECysw4rA
skXBE/ALhqYPP+LhUFAR18eft3/8TI4ebj9jeJVcrYivenaIB8DAzE96GP4Z zm/RBQaH3ouF
DYbfoQF5W4spJm5q9L0BF/Om3JKMw8ubc/P9kkPMqjyukjZjuF92rFnVWHhA l8496wd09R7+
K0d0VccldTJTyOQ4n88iy72Sl0C2/V9uCtiz9r+Vsv/Nan7/70rgW6ffiLP7 T/4FkFgR/PrE
HxFIlV38V0BSRZ/9OyDS5YhooKx83HqTyUB7AepmhtsKgVi2L8gfmtzsCzTv EhkgT+/4x6ot
yniLoZa9RttQVl3XMgIo1MVgITuyXcuBzkfDA0yk5kW0vuehd/SgB95C74a7 tLclaqiwWTeg
kJlqQXpcmImmYQfbh9FEIFGUhquhBgLQnRbYgBmM7WFoUJ4Q2/4R50gbE228 QfFiqkErakt5
SGPPATvtbwRxR00Y8UBvPyNr7ysz7LK6WNKogoOP3ArVA9udh+Rlb8hIyg3i JDnkRmKST9d/
FJSTUZl+OKOJFcxLm5UqmyWGcJfYYezNgEg95C1ElO5FfGCJLgyRiZyHshge OFIqEwSoP8I/
9CV9MtUbhT4jUiM7tyBSkyKLiIJqW6syWiGcDBWSkFIoUklgSvIoxQLIvkhh uW08d/8TXvZN
93/UO81GE88JN+ud/P6PlcBzB/sZO8AdYtSy6hoPuYbF+HDADMuZeGBFSQmg r749sEIruhak
gpZtvhaf7xRFiyR1/4BUiequEaip9HMgd3eylMex5dpDHoSmWMdnKBHCIv90 X7w0BX3nO5dY
RgVxuyp7vVup0PZkMoirYsQ4alMVlr09/GX/8PQCRuMpYnHlRJiX/fEVjm0t 3SS9W89uLnJf
zCkquKH2B5AUPJWyIZ0gmjP1J4yG0Z8v0v6Sk6jCHVz6pjsIq6wF/dPJsYnl JCm0MPkf6UOs
Adyv5hgJgiU9BQSscENsfozBwjtqM0hvcwHJ5FZvxHonHz/8XU6X6DaWHehG OpCMC7jjPCR3
qIP87GMOOeSQQw455JBDDjnkkEMOOeSQQw455JBDDjnkkEMOOeSQQw455JBD DjnkkEMOOeSw
Mvg/qrFy/gCgAAA=
--------------010608060505010600090503--
Re: Issue with building large libraries on Windows [message #526005 is a reply to message #525732] Thu, 08 April 2010 14:10 Go to previous messageGo to next message
Patrick Geremia is currently offline Patrick GeremiaFriend
Messages: 79
Registered: July 2009
Member
-------- Original Message --------
Subject: Re: Issue with building large libraries on Windows
From: dave russo <d-russo@ti.com>
To:
Date: Wed Apr 07 2010 17:32:28 GMT+0200 (CEST)
> Target filters must be modules that implement the ITargetFilter
> interface. The functions shown below (archive and mkCommandFile) must
> be in the .xs file of the module that implements ITargetFilter.
>
> See the attached example package basic.filter.
>
> On 4/7/2010 2:53 AM, Patrick Geremia wrote:
>> -------- Original Message --------
>> Subject: Re: Issue with building large libraries on Windows
>> From: dave russo <d-russo@ti.com>
>> To:
>> Date: Wed Oct 21 2009 02:04:37 GMT+0200 (CEST)
>>
>>> comments below
>>>
>>> Laurent Gauthier wrote:
>>>> Hi there,
>>>>
>>>> We are in the process of moving some of our software builds over to
>>>> RTSC, and are running into an issue that we think is related to very
>>>> long command lines on Windows.
>>>>
>>>> We create a library with over 400 C source files in it. The target is
>>>> a C64+ target.
>>>>
>>> [snip]
>>>>
>>>> To work around this problem our previous build system has a
>>>> work-around built-in that consist in passing to the archiver the list
>>>> of object files for the library in a file rather than on the command
>>>> line.
>>>>
>>>> Another work-around I have seen implemented in another build system
>>>> was a creation of the library archive in several pass (creation of
>>>> the archive with a first set of object files, and addition of more
>>>> object files in several increments) each of the passes having a
>>>> limited command line length.
>>>>
>>>> The questions we have are:
>>>> 1) Is there already a mechanism built-in to avoid this command line
>>>> limitation (apart from splitting our large library in several smaller
>>>> libs)? If so a pointer to how to do it would be appreciated.
>>>> 2) If there is no such mechanism, are there way to add such a
>>>> mechanism to RTSC as is, or would a change to RTSC be required...
>>>>
>>> Since every tool chain has unique options for getting around command
>>> line length limitations, the RTSC build system has to rely on the
>>> target to supply commands that can be executed. For example, the TI
>>> archiver supports a command file approach but the GNU archiver (as far
>>> as I know) does not - it requires the second workaround you describe
>>> above.
>>>
>>> Since RTSC targets can supply one or more arbitrary commands to
>>> perform the archiving, the problem you are experiencing can be solved
>>> in at least two ways:
>>> 1. fix the TI targets to use command files, or
>>> 2. create a "target filter" that replaces long lines with command
>>> files
>>>
>>> The first option is to fix the ti.targets.ITarget code to always use
>>> command files supported by the TI tool chain. Since ti.targets is
>>> currently bundled with XDCtools, this requires a re-release of
>>> XDCtools from TI and no doubt you don't want to wait for a new
>>> release. On the other hand, this fix should be added to the list of
>>> "bugs" to be addressed.
>>>
>>> The second option is something that is supported by the RTSC build
>>> engine and allows you to "filter" each command used as part of a
>>> build. In particular, you can write a small filter that changes each
>>> archive command (that would normally be added to the generated
>>> makefile) into a command that uses the -@ option.
>>>
>>> Reference docs for target filters are here:
>>> http://rtsc.eclipse.org/cdoc-tip/index.html#xdc/bld/ITargetF ilter.html
>>>
>>> I've created a small example that I believe works with the TI tool
>>> chain and I'll put it in the RTSC SVN repository. The core of the
>>> filter is the following XDCscript file which "implements" the
>>> xdc.bld.ITargetFilter interface:
>>>
>>> /*
>>> * ======== archive ========
>>> */
>>> function archive(container, lib, objList, archArgs, res)
>>> {
>>> /* replace ar command with one that uses the arguments in a file */
>>> res.cmds = mkCommandFile(container, res.cmds);
>>> }
>>>
>>> /*
>>> * ======== mkCommandFile ========
>>> */
>>> function mkCommandFile(archiveName, command)
>>> {
>>> var tokens = command.split(/\s+/);
>>> var cmd = tokens.shift() + " -@" + archiveName + ".cmd";
>>>
>>> /* create a command file to hold ar's command line options */
>>> var file = new java.io.BufferedWriter(
>>> new java.io.FileWriter(archiveName + ".cmd"));
>>>
>>> /* put archiver options in the file*/
>>> file.write(tokens.shift() + '\n');
>>>
>>> /* replace $@ token with actual archive name */
>>> tokens.shift(); /* should be == $@ */
>>> file.write(archiveName + '\n');
>>>
>>> /* finally, put all objects to go into the archive */
>>> for (var i = 0; i < tokens.length; i++) {
>>> file.write(tokens[i] + '\n');
>>> }
>>>
>>> file.flush();
>>>
>>> /* return new ar command to be placed in makefile */
>>> return (cmd);
>>> }
>>>
>>>
>> Dave,
>>
>> could you be a little bit more specific on how to implement 2) in
>> practice? I have hard-time understanding how to fit archive function
>> above in my config.bld (see attachment).
>>
Dave,

it worked like a charm! Thanks.
Do you happen to have readily available the filter for gcc i.e. suppling
and MRI script to ar (ar -M <mri_script>)?

--
Patrick Geremia
Texas Instruments (http://www.ti.com)
Phone: +33 4 93 22 26 33
Email: p-geremia@ti.com
Availability: http://meetwith.me/patrickgeremia
Re: Issue with building large libraries on Windows [message #526037 is a reply to message #526005] Thu, 08 April 2010 14:48 Go to previous messageGo to next message
Dave Russo is currently offline Dave RussoFriend
Messages: 172
Registered: July 2009
Senior Member
On 4/8/2010 7:10 AM, Patrick Geremia wrote:
>
> it worked like a charm! Thanks.
> Do you happen to have readily available the filter for gcc i.e. suppling
> and MRI script to ar (ar -M <mri_script>)?
>
I'm not familiar with MRI scripts but did find this post
http://old.nabble.com/Re%3A-Creating-a-static-lib-from-objec ts-and-another-static-lib-p16131684.html

It should be easy to create such a script but it might be better to
simply break the ar command into multiple commands; e.g.,
ar -ca lib.a foo1.o
ar -a lib.a foo2.o
:

This approach works with all the archivers I'm familiar with (ti, posix,
microsoft).
Re: Issue with building large libraries on Windows [message #526038 is a reply to message #526037] Thu, 08 April 2010 14:52 Go to previous messageGo to next message
Patrick Geremia is currently offline Patrick GeremiaFriend
Messages: 79
Registered: July 2009
Member
-------- Original Message --------
Subject: Re: Issue with building large libraries on Windows
From: dave russo <d-russo@ti.com>
To:
Date: Thu Apr 08 2010 16:48:37 GMT+0200 (CEST)
>
>
> On 4/8/2010 7:10 AM, Patrick Geremia wrote:
>>
>> it worked like a charm! Thanks.
>> Do you happen to have readily available the filter for gcc i.e. suppling
>> and MRI script to ar (ar -M <mri_script>)?
>>
> I'm not familiar with MRI scripts but did find this post
> http://old.nabble.com/Re%3A-Creating-a-static-lib-from-objec ts-and-another-static-lib-p16131684.html
>
>
> It should be easy to create such a script but it might be better to
> simply break the ar command into multiple commands; e.g.,
> ar -ca lib.a foo1.o
> ar -a lib.a foo2.o
> :
>
> This approach works with all the archivers I'm familiar with (ti,
> posix, microsoft).
Dave,

why don't you make it default or at least configurable in in RTSC?

--
Patrick Geremia
Texas Instruments (http://www.ti.com)
Phone: +33 4 93 22 26 33
Email: p-geremia@ti.com
Availability: http://meetwith.me/patrickgeremia
Re: Issue with building large libraries on Windows [message #526083 is a reply to message #526038] Thu, 08 April 2010 15:52 Go to previous message
Dave Russo is currently offline Dave RussoFriend
Messages: 172
Registered: July 2009
Senior Member
On 4/8/2010 7:52 AM, Patrick Geremia wrote:
[snip]
>>
>> This approach works with all the archivers I'm familiar with (ti,
>> posix, microsoft).
> Dave,
>
> why don't you make it default or at least configurable in in RTSC?
>
The RTSC build engine simply takes commands from provided by the target
and adds them to the makefile; there is no common method for creating an
archive or adding elements to the archive. So it's the target's
responsibility to not generate command lines too long for the host OS.

The problems you are experiencing are essentially "bugs" in the targets.
They should limit the length of the commands passed to the RTSC build
engine using what every technique is appropriate for the particular tool
chain.

Of course, we've been getting away with this because, over time,
Microsoft has dramatically increased the maximum command line length
from 128 bytes to many K bytes.

Patches to the targets are welcome.
Previous Topic:disabling dependencies generation
Next Topic:use of requires statement
Goto Forum:
  


Current Time: Sat Jun 23 18:13:15 GMT 2018

Powered by FUDForum. Page generated in 0.02517 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top