[gambit-list] Gambit Clang C/C++ status and compiler benchmarks?

[Adam]

Hi!

Does Gambit work perfectly with the Clang C/C++ compiler (compiling to
assembly which is compiled to native code, nothing LLVM), if so is it for
all Gambit versions, or since when?

I remember a benchmark of the time taken to, was it to compile
Gambit-generated C code, or was it execution time of Gambit-C code as
compiled by different GCC and Clang versions. What is the URL to that post
in the mailing list archive?

Thanks!
Adam

[Adam]

Hi list,

So I think I figured out the answer myself.

First, the previous benchmark that I recalled having read, is the "8)
Performance of GCC and CLANG when compiling Gambit" section in Gambit's
install.txt file (
https://github.com/gambit/gambit/blob/edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577
).

The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile
Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.

A lot has happened since Clang 3.1, which is the last Clang version covered
by install.txt today. The latest version today is 6.0. Ref.
https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the
default C/C++/Objective-C compiler choice in a handful operating systems,
for AMD64 and some more architectures.

Clang's language feature set is good, ref.
https://en.cppreference.com/w/cpp/compiler_support ,
https://clang.llvm.org/cxx_status.html .

And Clang's speed is decent, recent benchmarks tend to find that Clang and
GCC have a speed difference that's in the ballpark +-30%, up or down
depending on benchmark, e.g. ref.
https://stackoverflow.com/questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814
.

The term "LLVM" only denotes that Clang has an internal intermediary
language form, and the term LLVM has no other meaning in the direction of
executable bytecode like Java JIT VM:s. I.e. Clang is only a native
C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page
at http://www.llvm.org/ .

I have not tested yet but I do expect Clang to run Gambit stably and at a
totally-okay speed.

Adam

Post by Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to
assembly which is compiled to native code, nothing LLVM), if so is it for
all Gambit versions, or since when?
I remember a benchmark of the time taken to, was it to compile
Gambit-generated C code, or was it execution time of Gambit-C code as
compiled by different GCC and Clang versions. What is the URL to that post
in the mailing list archive?
Thanks!
Adam

[Marc Feeley]

GCC is still the best choice when compiling Gambit.

When configured with the usual

./configure --enable-single-host

the build time of a “make -j8” on my 4 core laptop is over 10x slower when using CLANG. As for speed of the generated code, CLANG produces code that is about 3x slower than GCC.

Here are the raw results, in seconds:

make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229

When configured with

./configure --enable-single-host --enable-c-opt

which uses -O2 rather than the default -O1, the results are also favourable for GCC. However the build times are closer and, interestingly, both compilers generate slightly slower code with -O2:

make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410

Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the "8) Performance of GCC and CLANG when compiling Gambit" section in Gambit's install.txt file (https://github.com/gambit/gambit/blob/edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.
A lot has happened since Clang 3.1, which is the last Clang version covered by install.txt today. The latest version today is 6.0. Ref. https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the default C/C++/Objective-C compiler choice in a handful operating systems, for AMD64 and some more architectures.
Clang's language feature set is good, ref. https://en.cppreference.com/w/cpp/compiler_support , https://clang.llvm.org/cxx_status.html .
And Clang's speed is decent, recent benchmarks tend to find that Clang and GCC have a speed difference that's in the ballpark +-30%, up or down depending on benchmark, e.g. ref. https://stackoverflow.com/questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .
The term "LLVM" only denotes that Clang has an internal intermediary language form, and the term LLVM has no other meaning in the direction of executable bytecode like Java JIT VM:s. I.e. Clang is only a native C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page at http://www.llvm.org/ .
I have not tested yet but I do expect Clang to run Gambit stably and at a totally-okay speed.
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to assembly which is compiled to native code, nothing LLVM), if so is it for all Gambit versions, or since when?
I remember a benchmark of the time taken to, was it to compile Gambit-generated C code, or was it execution time of Gambit-C code as compiled by different GCC and Clang versions. What is the URL to that post in the mailing list archive?
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Adam]

Hi Marc,

Wow, Clang is immensely slow indeed.

Do you have any idea if Clang's low performance could be circumvented using
Clang optimizer configuration, or, do you have any idea about qualities in
Clang's design that destine it to produce slow code for Gambit?

Maybe there could be a conversation with Clang to ask them to have a look
at why their compiler is performing so bad, maybe they would be interested
in speeding up things, I think overall they do have an ambition for high
performance.

Clang has become the bundled system-default C/C++/Objective C compiler for
a handful OS:es now, so there is a growing incentive to use it. All the
OS-bundled and packaged libraries on those platforms are Clang-compiled,
and mixing Clang- and GCC-compiled code (e.g. shared libraries) in one
executable, is not a very smooth experience today at least in my very
limited experience.

Would you be interested in crossposting an email to Clang's developer
mailing list (llvm-dev and maybe cfe-dev,
http://lists.llvm.org/mailman/listinfo) where you address Clang's
performance problem and suggest any reasons you see as plausible for why
they are so much after?

Adam

Post by Marc Feeley
GCC is still the best choice when compiling Gambit.
When configured with the usual
./configure --enable-single-host
the build time of a “make -j8” on my 4 core laptop is over 10x slower when
using CLANG. As for speed of the generated code, CLANG produces code that
is about 3x slower than GCC.
make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229
When configured with
./configure --enable-single-host --enable-c-opt
which uses -O2 rather than the default -O1, the results are also
favourable for GCC. However the build times are closer and, interestingly,
make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410
Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the "8)

Performance of GCC and CLANG when compiling Gambit" section in Gambit's
install.txt file (https://github.com/gambit/gambit/blob/
edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).

Post by Adam
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile

Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.

Post by Adam
A lot has happened since Clang 3.1, which is the last Clang version

covered by install.txt today. The latest version today is 6.0. Ref.
https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the
default C/C++/Objective-C compiler choice in a handful operating systems,
for AMD64 and some more architectures.

Post by Adam
Clang's language feature set is good, ref.

https://en.cppreference.com/w/cpp/compiler_support ,
https://clang.llvm.org/cxx_status.html .

Post by Adam
And Clang's speed is decent, recent benchmarks tend to find that Clang

and GCC have a speed difference that's in the ballpark +-30%, up or down
depending on benchmark, e.g. ref. https://stackoverflow.com/
questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .

Post by Adam
The term "LLVM" only denotes that Clang has an internal intermediary

language form, and the term LLVM has no other meaning in the direction of
executable bytecode like Java JIT VM:s. I.e. Clang is only a native
C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page
at http://www.llvm.org/ .

Post by Adam
I have not tested yet but I do expect Clang to run Gambit stably and at

a totally-okay speed.

Post by Adam
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to

assembly which is compiled to native code, nothing LLVM), if so is it for
all Gambit versions, or since when?

Post by Adam
I remember a benchmark of the time taken to, was it to compile

Gambit-generated C code, or was it execution time of Gambit-C code as
compiled by different GCC and Clang versions. What is the URL to that post
in the mailing list archive?

Post by Adam
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Marc Feeley]

I haven't experimented with clang very much. The performance gap with gcc is so wide that it seems like a very long shot to get it to perform at least as well as gcc on Gambit. gcc gives adequate performance so why look for better? As Voltaire said, “better is the enemy of good”.

I brought this performance issue up with the Apple LLVM team 3 years ago. At the time it was logged as rdar://problem/10739380 on their internal bug tracking system. I have poked them from time to time to see if there is any progress, and at this point I’ve lost interest in seeing a resolution. It is not on any critical path of mine.

If someone has the interest and time to followup with the developpers, be my guest.

Marc

Post by Adam
Hi Marc,
Wow, Clang is immensely slow indeed.
Do you have any idea if Clang's low performance could be circumvented using Clang optimizer configuration, or, do you have any idea about qualities in Clang's design that destine it to produce slow code for Gambit?
Maybe there could be a conversation with Clang to ask them to have a look at why their compiler is performing so bad, maybe they would be interested in speeding up things, I think overall they do have an ambition for high performance.
Clang has become the bundled system-default C/C++/Objective C compiler for a handful OS:es now, so there is a growing incentive to use it. All the OS-bundled and packaged libraries on those platforms are Clang-compiled, and mixing Clang- and GCC-compiled code (e.g. shared libraries) in one executable, is not a very smooth experience today at least in my very limited experience.
Would you be interested in crossposting an email to Clang's developer mailing list (llvm-dev and maybe cfe-dev, http://lists.llvm.org/mailman/listinfo) where you address Clang's performance problem and suggest any reasons you see as plausible for why they are so much after?
Adam
GCC is still the best choice when compiling Gambit.
When configured with the usual
./configure --enable-single-host
the build time of a “make -j8” on my 4 core laptop is over 10x slower when using CLANG. As for speed of the generated code, CLANG produces code that is about 3x slower than GCC.
make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229
When configured with
./configure --enable-single-host --enable-c-opt
make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410
Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the "8) Performance of GCC and CLANG when compiling Gambit" section in Gambit's install.txt file (https://github.com/gambit/gambit/blob/edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.
A lot has happened since Clang 3.1, which is the last Clang version covered by install.txt today. The latest version today is 6.0. Ref. https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the default C/C++/Objective-C compiler choice in a handful operating systems, for AMD64 and some more architectures.
Clang's language feature set is good, ref. https://en.cppreference.com/w/cpp/compiler_support , https://clang.llvm.org/cxx_status.html .
And Clang's speed is decent, recent benchmarks tend to find that Clang and GCC have a speed difference that's in the ballpark +-30%, up or down depending on benchmark, e.g. ref. https://stackoverflow.com/questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .
The term "LLVM" only denotes that Clang has an internal intermediary language form, and the term LLVM has no other meaning in the direction of executable bytecode like Java JIT VM:s. I.e. Clang is only a native C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page at http://www.llvm.org/ .
I have not tested yet but I do expect Clang to run Gambit stably and at a totally-okay speed.
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to assembly which is compiled to native code, nothing LLVM), if so is it for all Gambit versions, or since when?
I remember a benchmark of the time taken to, was it to compile Gambit-generated C code, or was it execution time of Gambit-C code as compiled by different GCC and Clang versions. What is the URL to that post in the mailing list archive?
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Adam]

Hi Marc,

First thanks for responding.

Yes GCC gives good performance however (my best understanding to date) is
that GCC- and Clang-compiled C++ shared libraries and other object code,
will not cooperate - libstdc++ (=GCC) and libc++ + libc++abi (=

This means that if you're on a Clang-compiled operating system, C++
packaged shared libraries will/may need to be recompiled to work with
Gambit.


I agree that GCC overall is fast, generally adequate, and has excellent
architecture support, and also Gambit compiles well on many other C/C++
compilers such as MSVC, right, so I agree that this sorts as a rather
peripheral issue.

It should be in LLVM's self-interest to compile Gambit code fast though.


I can't find 10739380 on https://bugs.llvm.org/ , do you remember the URL
where you submitted it?

Maybe the C preprocessor output of a Gambit program that illustrates the
problem would be instructive for the LLVM devs.


Thanks!
Adam

Post by Marc Feeley
I haven't experimented with clang very much. The performance gap with gcc
is so wide that it seems like a very long shot to get it to perform at
least as well as gcc on Gambit. gcc gives adequate performance so why look
for better? As Voltaire said, “better is the enemy of good”.
I brought this performance issue up with the Apple LLVM team 3 years ago.
At the time it was logged as rdar://problem/10739380 on their internal bug
tracking system. I have poked them from time to time to see if there is
any progress, and at this point I’ve lost interest in seeing a resolution.
It is not on any critical path of mine.
If someone has the interest and time to followup with the developpers, be my guest.
Marc

Post by Adam
Hi Marc,
Wow, Clang is immensely slow indeed.
Do you have any idea if Clang's low performance could be circumvented

using Clang optimizer configuration, or, do you have any idea about
qualities in Clang's design that destine it to produce slow code for Gambit?

Post by Adam
Maybe there could be a conversation with Clang to ask them to have a

look at why their compiler is performing so bad, maybe they would be
interested in speeding up things, I think overall they do have an ambition
for high performance.

Post by Adam
Clang has become the bundled system-default C/C++/Objective C compiler

for a handful OS:es now, so there is a growing incentive to use it. All the
OS-bundled and packaged libraries on those platforms are Clang-compiled,
and mixing Clang- and GCC-compiled code (e.g. shared libraries) in one
executable, is not a very smooth experience today at least in my very
limited experience.

Post by Adam
Would you be interested in crossposting an email to Clang's developer

mailing list (llvm-dev and maybe cfe-dev, http://lists.llvm.org/mailman/
listinfo) where you address Clang's performance problem and suggest any
reasons you see as plausible for why they are so much after?

Post by Adam
Adam
GCC is still the best choice when compiling Gambit.
When configured with the usual
./configure --enable-single-host
the build time of a “make -j8” on my 4 core laptop is over 10x slower

when using CLANG. As for speed of the generated code, CLANG produces code
that is about 3x slower than GCC.

Post by Adam
make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229
When configured with
./configure --enable-single-host --enable-c-opt
which uses -O2 rather than the default -O1, the results are also

favourable for GCC. However the build times are closer and, interestingly,

Post by Adam
make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410
Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the "8)

Performance of GCC and CLANG when compiling Gambit" section in Gambit's
install.txt file (https://github.com/gambit/gambit/blob/
edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).

Post by Adam

Post by Adam
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile

Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.

Post by Adam

Post by Adam
A lot has happened since Clang 3.1, which is the last Clang version

covered by install.txt today. The latest version today is 6.0. Ref.
https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the
default C/C++/Objective-C compiler choice in a handful operating systems,
for AMD64 and some more architectures.

Post by Adam

Post by Adam
Clang's language feature set is good, ref.

https://en.cppreference.com/w/cpp/compiler_support ,
https://clang.llvm.org/cxx_status.html .

Post by Adam

Post by Adam
And Clang's speed is decent, recent benchmarks tend to find that Clang

and GCC have a speed difference that's in the ballpark +-30%, up or down
depending on benchmark, e.g. ref. https://stackoverflow.com/
questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .

Post by Adam

Post by Adam
The term "LLVM" only denotes that Clang has an internal intermediary

language form, and the term LLVM has no other meaning in the direction of
executable bytecode like Java JIT VM:s. I.e. Clang is only a native
C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page
at http://www.llvm.org/ .

Post by Adam

Post by Adam
I have not tested yet but I do expect Clang to run Gambit stably and

at a totally-okay speed.

Post by Adam

Post by Adam
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to

assembly which is compiled to native code, nothing LLVM), if so is it for
all Gambit versions, or since when?

Post by Adam

Post by Adam
I remember a benchmark of the time taken to, was it to compile

Gambit-generated C code, or was it execution time of Gambit-C code as
compiled by different GCC and Clang versions. What is the URL to that post
in the mailing list archive?

Post by Adam

Post by Adam
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Marc Feeley]

As I said my efforts to “fix” clang have not been very fruitful. The issue 10739380 is on the Apple internal issue system.

Please note also that Gambit was designed to allow mixing code compiled by different C compilers. So you can compile the runtime system using clang and your application using gcc, or the other way around. There may be issues on some systems, particularly if the FFI or C++ is used. I’ve used this on macOS, in fact the Gambit prebuilt installer is built with GNU gcc and gsc will work on the user’s machine even if only Xcode is installed there.

Marc

Post by Adam
Hi Marc,
First thanks for responding.
Yes GCC gives good performance however (my best understanding to date) is that GCC- and Clang-compiled C++ shared libraries and other object code, will not cooperate - libstdc++ (=GCC) and libc++ + libc++abi (=
This means that if you're on a Clang-compiled operating system, C++ packaged shared libraries will/may need to be recompiled to work with Gambit.
I agree that GCC overall is fast, generally adequate, and has excellent architecture support, and also Gambit compiles well on many other C/C++ compilers such as MSVC, right, so I agree that this sorts as a rather peripheral issue.
It should be in LLVM's self-interest to compile Gambit code fast though.
I can't find 10739380 on https://bugs.llvm.org/ , do you remember the URL where you submitted it?
Maybe the C preprocessor output of a Gambit program that illustrates the problem would be instructive for the LLVM devs.
Thanks!
Adam
I haven't experimented with clang very much. The performance gap with gcc is so wide that it seems like a very long shot to get it to perform at least as well as gcc on Gambit. gcc gives adequate performance so why look for better? As Voltaire said, “better is the enemy of good”.
I brought this performance issue up with the Apple LLVM team 3 years ago. At the time it was logged as rdar://problem/10739380 on their internal bug tracking system. I have poked them from time to time to see if there is any progress, and at this point I’ve lost interest in seeing a resolution. It is not on any critical path of mine.
If someone has the interest and time to followup with the developpers, be my guest.
Marc

Post by Adam
Hi Marc,
Wow, Clang is immensely slow indeed.
Do you have any idea if Clang's low performance could be circumvented using Clang optimizer configuration, or, do you have any idea about qualities in Clang's design that destine it to produce slow code for Gambit?
Maybe there could be a conversation with Clang to ask them to have a look at why their compiler is performing so bad, maybe they would be interested in speeding up things, I think overall they do have an ambition for high performance.
Clang has become the bundled system-default C/C++/Objective C compiler for a handful OS:es now, so there is a growing incentive to use it. All the OS-bundled and packaged libraries on those platforms are Clang-compiled, and mixing Clang- and GCC-compiled code (e.g. shared libraries) in one executable, is not a very smooth experience today at least in my very limited experience.
Would you be interested in crossposting an email to Clang's developer mailing list (llvm-dev and maybe cfe-dev, http://lists.llvm.org/mailman/listinfo) where you address Clang's performance problem and suggest any reasons you see as plausible for why they are so much after?
Adam
GCC is still the best choice when compiling Gambit.
When configured with the usual
./configure --enable-single-host
the build time of a “make -j8” on my 4 core laptop is over 10x slower when using CLANG. As for speed of the generated code, CLANG produces code that is about 3x slower than GCC.
make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229
When configured with
./configure --enable-single-host --enable-c-opt
make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410
Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the "8) Performance of GCC and CLANG when compiling Gambit" section in Gambit's install.txt file (https://github.com/gambit/gambit/blob/edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even compile Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.
A lot has happened since Clang 3.1, which is the last Clang version covered by install.txt today. The latest version today is 6.0. Ref. https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the default C/C++/Objective-C compiler choice in a handful operating systems, for AMD64 and some more architectures.
Clang's language feature set is good, ref. https://en.cppreference.com/w/cpp/compiler_support , https://clang.llvm.org/cxx_status.html .
And Clang's speed is decent, recent benchmarks tend to find that Clang and GCC have a speed difference that's in the ballpark +-30%, up or down depending on benchmark, e.g. ref. https://stackoverflow.com/questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .
The term "LLVM" only denotes that Clang has an internal intermediary language form, and the term LLVM has no other meaning in the direction of executable bytecode like Java JIT VM:s. I.e. Clang is only a native C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page at http://www.llvm.org/ .
I have not tested yet but I do expect Clang to run Gambit stably and at a totally-okay speed.
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling to assembly which is compiled to native code, nothing LLVM), if so is it for all Gambit versions, or since when?
I remember a benchmark of the time taken to, was it to compile Gambit-generated C code, or was it execution time of Gambit-C code as compiled by different GCC and Clang versions. What is the URL to that post in the mailing list archive?
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Adam]

Hi Marc,

As I said my efforts to “fix” clang have not been very fruitful. The
issue 10739380 is on the Apple internal issue system.

Aha. I can't find that anywhere. I guess the "llvm-dev" mailing list is the
right place.

Please note also that Gambit was designed to allow mixing code compiled by
different C compilers. So you can compile the runtime system using clang
and your application using gcc, or the other way around.

Right.

What causes this lower speed should likely boil down to that the way Gambit
implements trampolines and its whole single host / calling convention model
in C, that for some reason LLVM fails to make those exact logics fast.

I deduce that reasoning from the general observation that GCC and LLVM
these days generally are in about the same performance ballpark, if the
reports I read were correct - my thought then is that LLVM cannot possibly
compile all of Gambit's runtime slower than GCC, as those parts of Gambit
are fairly similar in nature to other C applications [and so what LLVM
would need to optimize should be Gambit's trampolines/single host/calling
conventions that LLVM].

Do you think that line of reasoning would make sense?

In this case the understanding would be that it's the application logic in
the user-compiled Scheme modules that need to be optimized. So just some
simple Scheme program, that maybe even doesn't invoke the runtime at all
but maybe is implemented as two modules as to trig the trampolines a lot,
should be representative as C code that LLVM is slow at now and would need
to speed up to make Clang attractive for Gambit.

There may be issues on some systems, particularly if the FFI or C++ is
used.

Right.

I’ve used this on macOS, in fact the Gambit prebuilt installer is built
with GNU gcc and gsc will work on the user’s machine even if only Xcode is
installed there.

Ok!

Yes I understand as long as all is C only, they interoperate well. Clang
has had as implementation goal to be as compatible with GCC as possible on
all levels relevant to users.

Adam

Post by Adam
Hi Marc,
First thanks for responding.
Yes GCC gives good performance however (my best understanding to date)

is that GCC- and Clang-compiled C++ shared libraries and other object code,
will not cooperate - libstdc++ (=GCC) and libc++ + libc++abi (=

Post by Adam
This means that if you're on a Clang-compiled operating system, C++

packaged shared libraries will/may need to be recompiled to work with
Gambit.

Post by Adam
I agree that GCC overall is fast, generally adequate, and has excellent

architecture support, and also Gambit compiles well on many other C/C++
compilers such as MSVC, right, so I agree that this sorts as a rather
peripheral issue.

Post by Adam
It should be in LLVM's self-interest to compile Gambit code fast though.
I can't find 10739380 on https://bugs.llvm.org/ , do you remember the

URL where you submitted it?

Post by Adam
Maybe the C preprocessor output of a Gambit program that illustrates the

problem would be instructive for the LLVM devs.

Post by Adam
Thanks!
Adam
I haven't experimented with clang very much. The performance gap with

gcc is so wide that it seems like a very long shot to get it to perform at
least as well as gcc on Gambit. gcc gives adequate performance so why look
for better? As Voltaire said, “better is the enemy of good”.

Post by Adam
I brought this performance issue up with the Apple LLVM team 3 years

ago. At the time it was logged as rdar://problem/10739380 on their
internal bug tracking system. I have poked them from time to time to see
if there is any progress, and at this point I’ve lost interest in seeing a
resolution. It is not on any critical path of mine.

Post by Adam
If someone has the interest and time to followup with the developpers,

be my guest.

Post by Adam
Marc

Post by Adam
Hi Marc,
Wow, Clang is immensely slow indeed.
Do you have any idea if Clang's low performance could be circumvented

using Clang optimizer configuration, or, do you have any idea about
qualities in Clang's design that destine it to produce slow code for Gambit?

Post by Adam

Post by Adam
Maybe there could be a conversation with Clang to ask them to have a

look at why their compiler is performing so bad, maybe they would be
interested in speeding up things, I think overall they do have an ambition
for high performance.

Post by Adam

Post by Adam
Clang has become the bundled system-default C/C++/Objective C compiler

for a handful OS:es now, so there is a growing incentive to use it. All the
OS-bundled and packaged libraries on those platforms are Clang-compiled,
and mixing Clang- and GCC-compiled code (e.g. shared libraries) in one
executable, is not a very smooth experience today at least in my very
limited experience.

Post by Adam

Post by Adam
Would you be interested in crossposting an email to Clang's developer

mailing list (llvm-dev and maybe cfe-dev, http://lists.llvm.org/mailman/
listinfo) where you address Clang's performance problem and suggest any
reasons you see as plausible for why they are so much after?

Post by Adam

Post by Adam
Adam
GCC is still the best choice when compiling Gambit.
When configured with the usual
./configure --enable-single-host
the build time of a “make -j8” on my 4 core laptop is over 10x slower

when using CLANG. As for speed of the generated code, CLANG produces code
that is about 3x slower than GCC.

Post by Adam

Post by Adam
make -j8 test4
GCC 8.1.0 39.566 1.167
CLANG 802.0.38 426.371 4.229
When configured with
./configure --enable-single-host --enable-c-opt
which uses -O2 rather than the default -O1, the results are also

favourable for GCC. However the build times are closer and, interestingly,

Post by Adam

Post by Adam
make -j8 test4
GCC 8.1.0 193.560 1.218
CLANG 802.0.38 229.945 4.410
Marc

Post by Adam
Hi list,
So I think I figured out the answer myself.
First, the previous benchmark that I recalled having read, is the

"8) Performance of GCC and CLANG when compiling Gambit" section in Gambit's
install.txt file (https://github.com/gambit/gambit/blob/
edac3c75f0d93f4f56a39c2b90621511f096dd72/INSTALL.txt#L577).

Post by Adam

Post by Adam

Post by Adam
The essence here is that Clang 2.6, 2.7 and 2.8 wouldn't even

compile Gambit, and 2.9-3.1 would compile Gambit but be fairly slow.

Post by Adam

Post by Adam

Post by Adam
A lot has happened since Clang 3.1, which is the last Clang version

covered by install.txt today. The latest version today is 6.0. Ref.
https://en.wikipedia.org/wiki/Clang#Status_history . Clang is now the
default C/C++/Objective-C compiler choice in a handful operating systems,
for AMD64 and some more architectures.

Post by Adam

Post by Adam

Post by Adam
Clang's language feature set is good, ref.

https://en.cppreference.com/w/cpp/compiler_support ,
https://clang.llvm.org/cxx_status.html .

Post by Adam

Post by Adam

Post by Adam
And Clang's speed is decent, recent benchmarks tend to find that

Clang and GCC have a speed difference that's in the ballpark +-30%, up or
down depending on benchmark, e.g. ref. https://stackoverflow.com/
questions/3187414/clang-vs-gcc-which-produces-better-binaries#15043814 .

Post by Adam

Post by Adam

Post by Adam
The term "LLVM" only denotes that Clang has an internal intermediary

language form, and the term LLVM has no other meaning in the direction of
executable bytecode like Java JIT VM:s. I.e. Clang is only a native
C/C++/Obj-C compiler. A list of the Clang/LLVM projects is on the main page
at http://www.llvm.org/ .

Post by Adam

Post by Adam

Post by Adam
I have not tested yet but I do expect Clang to run Gambit stably and

at a totally-okay speed.

Post by Adam

Post by Adam

Post by Adam
Adam
Hi!
Does Gambit work perfectly with the Clang C/C++ compiler (compiling

to assembly which is compiled to native code, nothing LLVM), if so is it
for all Gambit versions, or since when?

Post by Adam

Post by Adam

Post by Adam
I remember a benchmark of the time taken to, was it to compile

Gambit-generated C code, or was it execution time of Gambit-C code as
compiled by different GCC and Clang versions. What is the URL to that post
in the mailing list archive?

Post by Adam

Post by Adam

Post by Adam
Thanks!
Adam
_______________________________________________
Gambit-list mailing list
https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

[Marc Feeley]

What causes this lower speed should likely boil down to that the way Gambit implements trampolines and its whole single host / calling convention model in C, that for some reason LLVM fails to make those exact logics fast.
I deduce that reasoning from the general observation that GCC and LLVM these days generally are in about the same performance ballpark, if the reports I read were correct - my thought then is that LLVM cannot possibly compile all of Gambit's runtime slower than GCC, as those parts of Gambit are fairly similar in nature to other C applications [and so what LLVM would need to optimize should be Gambit's trampolines/single host/calling conventions that LLVM].
Do you think that line of reasoning would make sense?

It is possible that the difference in speed is due to clang not compiling the trampoline mechanism efficiently. But this can be due to a variety of underlying issues…

- not optimizing value labels and the “goto *…” statement
- not performing an optimization when the code is too large or irreducible (not having a structured control flow)
- not performing good register allocation when the code’s control flow is unstructured

If you are interested in solving this issue, I suggest you perform some experiments with gcc and clang. It shouldn’t take more than a few minutes. Try compiling your code with and without --enable-single-host . Please report your results back to the mailing list.

Marc