distcc

Has anybody tried distributed compiling for e2studio? I'm experiencing very slow builds on my I7 laptop and was wondering if this could be helpful.

 

Update July 22:

I've exported a stripped down version of my project which builds in 45 seconds on my setup without modifications. It can be downloaded here.

  • I sometimes have slow builds, it's because the compiler was/is compiling all the FIT modules every time. One of the moderators on the forum helped me figure out how to stop that. I can't remember what the solution was though. Maybe search the forum on 'slow compile' or something like that. My post might still be out there. Also, my coworker has no issues with speed in compiling. He has a much better PC than me with more resources. So maybe your PC is partly to blame too.
  • Just curious - how slow is "very slow"? What are your laptop's hardware specs (CPU, installed memory, storage device(s) and interface)? Might be easier to solve that issue than add support for distcc to Eclipse and (possibly) CC-RX support to distcc.
  • In reply to mickeyJoe:

    Thanks for the suggestions. Not sure if my PC is the bottleneck here. I'm running an Intel(R) Core(TM) i7-4600M CPU @ 2.90GHz with 8GB of RAM a Samsung EVO 840 SSD.

    I only just started programming on the Synergy platform, and already a complete rebuild takes about 10 minutes.

    I just ran a quick test by changing something whitespace in a single xxx_thread_entry.c file and pressing build. The first 34 seconds, e2studio.exe uses most of the CPU, while the progress tab reports the generation of a multitude of makefiles. Then the make.exe process takes over, consuming about a third of my CPU. The console shows "make -j4 all" for about 20 seconds before the actual gcc processes launch. None of them show any significant CPU usage and run very short. The total CPU usage is 100% now. The complete build took 1:39 minute on my stopwatch, 1:05 minute in the console.
  • In reply to AnthonyJenkins:

    You're right. It doesn't make sense to put lots of effort into making distcc work, since apparently gcc isn't my problem.
  • In reply to Joost:

    Joost
    Thanks for the suggestions. Not sure if my PC is the bottleneck here. I'm running an Intel(R) Core(TM) i7-4600M CPU @ 2.90GHz with 8GB of RAM a Samsung EVO 840 SSD.

    Looks comparable to mine (i7-3540 @ 3.00GHz, 8GB RAM).
     

    Joost
    I just ran a quick test by changing something whitespace in a single xxx_thread_entry.c file and pressing build. The first 34 seconds, e2studio.exe uses most of the CPU, while the progress tab reports the generation of a multitude of makefiles.

    Yeah that seems weird.  Makefile generation shouldn't take that long.  If the e2Studio process dominates the CPU cycles, I assume they're doing it in Java.

    I Googled "e2Studio slow" and found one (older) Renesas Rulz thread that suggests moving e2Studio and utility paths to the beginning of the PATH environment variable.  This would be significant if your PATH includes network folders which are slow to access.

    Joost
    Then the make.exe process takes over, consuming about a third of my CPU. The console shows "make -j4 all" for about 20 seconds before the actual gcc processes launch.

    This is (probably) make.exe resolving dependency order in the stuff it has to build.

    Joost
    None of them show any significant CPU usage and run very short.

    I assume "them" = "the gcc processes"?  Are your source files and/or e2Studio installation on a network share?

    Joost
    The total CPU usage is 100% now. The complete build took 1:39 minute on my stopwatch, 1:05 minute in the console.

  • In reply to AnthonyJenkins:

    No network folders are used. Just a local Git repo.

    The makefile generation delay has improved a lot after my latest reboot, so I'll call that one resolved.

    What's left is the delay between "make -j4 all" and the actual work being done. I have a pre-build step, which echoes the Git SHA to a header file. Then the main-build starts. Each step lags for about 15 seconds before anything happens, which makes a simple incremental build take about 50 seconds.
  • In reply to Joost:

    Joost
    No network folders are used. Just a local Git repo.

    Okay I'll rule that out.

    Joost
    The makefile generation delay has improved a lot after my latest reboot, so I'll call that one resolved.

    Makefile generation really should only happen once when you first build your project.  The generated makefiles are then read by 'make -j4 all' (which means "use up to 4 processor threads to build the 'all' target").  After the first build, e2Studio skips makefile generation, so I'm sure that slowness is still there.

    Joost
    What's left is the delay between "make -j4 all" and the actual work being done. I have a pre-build step, which echoes the Git SHA to a header file. Then the main-build starts. Each step lags for about 15 seconds before anything happens, which makes a simple incremental build take about 50 seconds.

    Sounds network-y to me...often (not specifically with e2Studio) I'll see a bad DNS lookup hang a process, finally timing out before the OS tries the next DNS server (which succeeds).  So the make.exe process is what's using lots of CPU, according to Task Manager?  Can you change your make command to turn on verbose logging?

    1. Right-click on project in Project Explorer window on left and select Properties....
    2. Select C/C++ Build in left panel.
    3. Select Builder Settings tab in main panel.
    4. In Build command: text field, change

               make -j4

      to

               make -j4 --debug=j,i --trace
    5. Click Apply and Close.
    6. Build project.

    This'll print the processes that make.exe forks to build your project, and should include build output that is normally suppressed in the makefiles with the '@' prefix to build commands.  Hopefully you'll see a process that make.exe starts that appears to hang for several seconds with each target it tries to build.

    Reference: GNU make - Options Summary

  • In reply to AnthonyJenkins:

    I just tried your suggestion about the make parameters. I ended up with a build log file that is 148377 lines long. Is this normal?

    The 3 make processes that start sequentially each get stuck at the part where they read the makefiles:

    20:35:39 **** Incremental Build of configuration Debug for project P0144_HMI ****
    make --debug=j,i --trace -j4 all
    GNU Make 4.0
    Built for Windows32
    Copyright (C) 1988-2013 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <gnu.org/.../gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Reading makefiles...

    Stuck for a while...

    Updating goal targets....
    File 'all' does not exist.
    Must remake target 'all'.
    Need a job token; we don't have children
    makefile:83: target 'all' does not exist
    make --no-print-directory pre-build
    CreateProcess(C:\Renesas\Synergy\e2studio_v7.3.0_ssp_v1.6.0\Utilities\make.exe,make --no-print-directory pre-build,...)
    Putting child 0013B410 (all) PID 37863488 on the chain.
    Live child 0013B410 (all) PID 37863488
    GNU Make 4.0
    Built for Windows32
    Copyright (C) 1988-2013 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <gnu.org/.../gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Jobserver client (semaphore gmake_semaphore_14272)
    Reading makefiles...

    Stuck for a while...

    Updating goal targets....
    File 'pre-build' does not exist.
    Must remake target 'pre-build'.
    CreateProcess(C:\Program Files\Git\cmd\git.exe,git describe --abbrev=4 --dirty --always --tags,...)
    CreateProcess(C:\Program Files\Git\cmd\git.exe,git rev-parse HEAD,...)
    Need a job token; we don't have children
    makefile:151: target 'pre-build' does not exist
    Creating temporary batch file C:\Users\Joost\AppData\Local\Temp\make2656-1.bat
    Batch file contents:
    @echo off
    echo #define GIT_VERSION "9df6-dirty" > ../src/git.h && echo #define GIT_SHA "9df602ec0ffbac1b44b3bf892ee8f60c85825088" >> ../src/git.h
    echo #define GIT_VERSION "9df6-dirty" > ../src/git.h && echo #define GIT_SHA "9df602ec0ffbac1b44b3bf892ee8f60c85825088" >> ../src/git.h
    CreateProcess(C:\Users\Joost\AppData\Local\Temp\make2656-1.bat,C:\Users\Joost\AppData\Local\Temp\make2656-1.bat,...)
    Putting child 00AAB398 (pre-build) PID 40845128 on the chain.
    Live child 00AAB398 (pre-build) PID 40845128
    Reaping winning child 00AAB398 PID 40845128
    Cleaning up temp batch file C:\Users\Joost\AppData\Local\Temp\make2656-1.bat
    Creating temporary batch file C:\Users\Joost\AppData\Local\Temp\make2656-2.bat
    Batch file contents:
    @echo off
    echo ' '
    echo ' '
    CreateProcess(C:\Users\Joost\AppData\Local\Temp\make2656-2.bat,C:\Users\Joost\AppData\Local\Temp\make2656-2.bat,...)
    Live child 00AAB398 (pre-build) PID 40843928
    ' '
    Reaping winning child 00AAB398 PID 40843928
    Cleaning up temp batch file C:\Users\Joost\AppData\Local\Temp\make2656-2.bat
    Removing child 00AAB398 PID 40843928 from chain.
    Reaping winning child 0013B410 PID 37863488
    make --no-print-directory main-build
    CreateProcess(C:\Renesas\Synergy\e2studio_v7.3.0_ssp_v1.6.0\Utilities\make.exe,make --no-print-directory main-build,...)
    Live child 0013B410 (all) PID 37864208
    GNU Make 4.0
    Built for Windows32
    Copyright (C) 1988-2013 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <gnu.org/.../gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Jobserver client (semaphore gmake_semaphore_14272)
    Reading makefiles...

    Stuck for a while...

    Updating goal targets....
     File 'main-build' does not exist.
      File 'P0144_HMI.elf' does not exist.

    The rest of the file contains 4653 occurrences of a line that looks similar to:

    Looking for an implicit rule for 'synergy/ssp_supplemental/touch_drivers/touch_panel_chip_sx8654/sf_touch_panel_chip_sx8654.o'.

    followed by a list of attempts to try pattern rules or implicit prerequisites for each occurrence. Ultimately, the build fails with 4 errors in this configuration, but I can't really tell why.

  • In reply to Joost:

    Yeah reading a local makefile shouldn't take that long...

    I tried creating 500 empty makefiles and one Makefile that included all 500; make.exe read all 501 makefiles and built my simple target ("touch foo.txt") in about 0.5 seconds.

    $ seq 1 500 | while read i; do touch "makefile-${i}.mk"; echo "include makefile-${i}.mk" >> Makefile; done
    $ cat >> Makefile <<EOF

    all:
    touch "foo.txt"
    EOF
    $ make --debug=i,j all
    GNU Make 4.2.1
    Built for x86_64-pc-msys
    Copyright (C) 1988-2016 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Reading makefiles...
    Updating makefiles....
    Updating goal targets....
     File 'all' does not exist.
    Must remake target 'all'.
    touch "foo.txt"
    Putting child 0x600240670 (all) PID 2679 on the chain.
    Live child 0x600240670 (all) PID 2679
    Reaping winning child 0x600240670 PID 2679
    Removing child 0x600240670 PID 2679 from chain.
    Successfully remade target file 'all'.

    How many makefiles (usually files with a ".mk" suffix") are in your project?  There's gotta be one or more it's getting stuck trying to read...

    You could try changing --debug=i,j to --debug=a to really turn up the debug verbosity.  You're just waiting to see a debug line that shows what make.exe is waiting on; you can kill the build after that.

  • In reply to AnthonyJenkins:

    The --debug=a parameter gave too much output. I tried ---debug=v and found something. There are  43 .mk files handled by make, followed by 2090 .d files. All goes pretty quick until the last few. Here's a snippet:

    Reading makefile 'src/hal_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/hmi_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/io_board_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/safety_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/water_level_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile '../makefile.defs' (search path) (don't care) (no ~ expansion)...
    Reading makefile '../makefile.targets' (search path) (don't care) (no ~ expansion)...
    Updating goal targets....
    Considering target file 'pre-build'.
    File 'pre-build' does not exist.
    Finished prerequisites of target file 'pre-build'.
    Must remake target 'pre-build'.
    Main thread handle = 000001A8
    echo #define GIT_VERSION "9df6-dirty" > ../src/git.h && echo #define GIT_SHA "9df602ec0ffbac1b44b3bf892ee8f60c85825088" >> ../src/git.h
    Recipe of 'pre-build' is being run.
    ' '
    Considering target file 'pre-build'.
    File 'pre-build' was considered already.
    make --no-print-directory main-build
    GNU Make 4.0
    Built for Windows32
    Copyright (C) 1988-2013 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <gnu.org/.../gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Reading makefiles...

    The process seems to wait at the safety_thread_entry.d line, however if I cancel the build at this point, the last lines of the console look like this:

    Reading makefile 'src/io_board_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/safety_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/water_level_thread_entry.d' (search path) (d

    Command canceled

    12:12:40 Build Cancelled. (took 2s.347ms)

    As if the build was stuck at the letter d from (don't care).

    If I exclude the safety_thread_entry.c file from the build, the process just waits at the next line:

    Reading makefile 'src/io_board_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile 'src/water_level_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile '../makefile.defs' (search path) (don't care
    Command canceled

    12:18:41 Build Cancelled. (took 2s.381ms)

    If I exclude that as well, the process just waits at makefile.defs:

    Reading makefile 'src/io_board_thread_entry.d' (search path) (don't care) (no ~ expansion)...
    Reading makefile '../makefile.defs' (search path) (don't care) (no ~ expansion)...
    Reading makefile '../makefile.targets' (search path) (don't care) (no ~ exp
    Command canceled

    12:20:58 Build Cancelled. (took 2s.931ms)

  • In reply to Joost:

    Well I'm officially stumped.  One last thing I'd ask you to try is to tell make.exe to not fork any parallel build processes by changing "-j4" to "-j1" and see where it hangs.  I don't see why Windows would do this, but one of the 4 build threads may be getting hung and what you're seeing is the output of the 3 "okay" build threads, which eventually hang waiting for the stuck thread.  But that doesn't make sense to me, particularly the console output hanging...  -j1 is standard practice in debugging GNU make build issues anyway, so we'll see what it does.

    The *.d files are dependency makefiles; they describe the dependencies between a source file and the headers it #include's.  They should be generated/updated at the start of every build.

  • In reply to AnthonyJenkins:

    Same result. One more thing I'll try before proceding to a state of acceptance is running make from a powershell terminal.

    Anyway, thanks for your help.
  • In reply to Joost:

    Well the hope with running single-threaded is that you get a clearer picture of exactly what GNU make is hanging on.

    Sorry I couldn't be more help...
  • In reply to AnthonyJenkins:

    Do you have anti-virus software running? I know that slows mine down to a crawl. I also don't use the make or sh "Utilities" that come with e2 studio . I believe the ones from git-bash run much faster, even if I don't disable anti virus.
  • In reply to cowens:

    I did disable the anti-virus as it uses more CPU than the build processes themselves. The build runs a lot faster, but the lag caused by make remains. How did you get e2studio to use the git-bash utilities?