HOWTO LAPACK/CLAPACK for Windows

Open discussion regarding features, bugs, issues, vendors, etc.

Did it work?

Yes
39
75%
No
13
25%
 
Total votes : 52

HOWTO LAPACK/CLAPACK for Windows

Postby admin » Tue Aug 11, 2009 2:25 pm

This is the first attempt of a HOWTO that I'm writing on using LAPACK with Visual Studio. In doing so, I've made the assumption that the reader, although probably an expert in some other discipline that requires a linear equation solver, is a novice when it comes to LAPACK -- just as I was when I started. For this reason, I've tried to explain the whys rather than dispense instructions.

Thanks to Evgenii Rudnyi and Mark Hoemmen at http://groups.google.com/group/matrixprogramming their early suggestions and the reading the draft of this HOWTO. Any inaccuracies in this document are mine. Use with care.


###########################################

Part 1: Compiling GotoBLAS
(Cygwin required -- http://cygwin.org/ )

###########################################

Preamble
LAPACK [http://www.netlib.org/lapack/] is designed as a two-tiered Fortran library, comprising higher level subroutines and "lower-level Basic Linear Algebra Subprograms (BLAS) in order to effectively exploit the caches on modern cache-based architectures" [http://en.wikipedia.org/wiki/LAPACK]. For reference purposes, the LAPACK installation provides a(n untuned) version of the BLAS which is not optimized for any architecture. This reference BLAS implementation may be orders of magnitude slower than optimized implementations, for matrix factorizations and other computationally intensive matrix operations. Optimized implementations the BLAS are available from a number of vendors and projects such as: Intel (commercial), AMD, and ATLAS, but my favorite is GotoBLAS because it is:

- currently the fastest BLAS implementation
[see research report "Choosing the optimal BLAS and LAPACK library" at http://tinyurl.com/m6lmqs ]
- trivial to compile
- is free (available without cost) for research/academic use.

You can read about sole author of this library Kazushige Goto (pronounced "Goat toe") at http://en.wikipedia.org/wiki/Kazushige_Goto

[Links to other BLAS implementations]
MKL - http://software.intel.com/en-us/intel-mkl/
AMD (AMCL) -- http://developer.amd.com/cpu/Libraries/ ... fault.aspx
ATLAS -- http://math-atlas.sourceforge.net/ (I was unable to build this library. I found the instructions too lengthy, complicated, and often ambiguous. Nevertheless, the goal of the ATLAS project is an admirable one. It aims to, by self-discovery, automatically generate an optimized BLAS library. However, its performance trails that of other libraries http://en.wikipedia.org/wiki/Automatica ... a_Software )

Compiling GotoBLAS
The GotoBLAS source is available from http://www.tacc.utexas.edu/resources/software/#blas (there's short registration
form to fill), and can be compiled for Windows with Cygwin. No changes need to be made to GotoBLAS' config file
Makefile.rule, unless a particular compiler is preferred. Happily, the config file automatically enables multithreading if more than one processor is available.

1. Download and extract the GotoBLAS source to any directory of choice, and make any desired changes to the config file (the default option should also work well).

2. In Cygwin, "cd" to the top-level directory containing the source, and type "make"

3. The result of this process should be a file libgoto_<processor_class>-r<gotoblas_version_number>.a, and a (symbolic) link libgoto.a pointing to this file. (For example, libgoto_banias-r1.26.a)

4. [Edit:] Now for the final part, building a Windows library (*.lib) and dll for GotoBLAS. This is best done in what's called the "Visual Studio command prompt". The Visual Studio command prompt is basically a version of of the command prompt (cmd) that knows the locations of all the Visual Studio tools e.g. compiler, linker.

- Add the path to the Cygwin binaries to your Windows PATH variable. (On my machine, this path is C:\cygwin\bin).
(Refer to the section "Adding or Editing Environment Variables" of the following page http://vlaurie.com/computers2/Articles/environment.htm for help on how to do this.)

- Launch the Visual Studio command prompt from:
Start Menu -> Programs -> Microsoft Visual Studio -> Visual Studio Tools -> Visual Studio Command prompt

- In the Visual Studio command prompt, change directory to the "exports" subdirectory of GotoBLAS and run the command "make dll". The result should be something like: libgoto_<processor_class>-r<gotoblas_version_number>.lib and libgoto_<processor_class>-r<gotoblas_version_number>.dll . For example: libgoto_banias-r1.26.dll, or libgoto_northwood-r1.26.dll

Final note:
* Windows needs to be told where to find this dll, else you will get a serious error when you try to run your program. There are several ways to do accomplish this. One, is to add the location of the dll to the PATH environment variable. Another is to simply copy the dll to the Windows\system32 folder. I did the later.

For more information, refer to Microsoft guidelines on Search Path Used by Windows to Locate a DLL

###########################################

Part 2: Using LAPACK subroutines in a Visual (Studio) C/C++ Project -- Part 1

###########################################

1. Download the LAPACK source or precompiled binaries from http://icl.cs.utk.edu/lapack-for-windows/ . Sources have been and can be can be compiled into debug and release versions. File names of the precomputed debug libraries end with the letter "d" e.g. BLASd.lib and lapackd.lib (in comparison to the release versions BLAS.lib and lapack.lib).

2. If you chose to compile LAPACK, you will require a Fortran compiler and libraries. I recommend the Intel Fortran compiler (distributed with libraries) [http://software.intel.com/en-us/articles/intel-software-evaluation-center/].

3. Obtain a tuned version of BLAS for your machine (refer to "Compiling GotoBLAS").

4. Create a Visual Studio project with the following sample C program:
http://www.cs.rochester.edu/~bh/cs400/using_lapack.html

5. rename the prototypes in the above program to
void dgesv_( )
void dgels_( )

to
extern "C" void dgesv_( )
extern "C" void dgels_( )

6. Add the following libraries to the Visual Studio project settings, under Linker -> Input -> Additional Dependencies. For example, on my laptop:
libgoto_banias-r1.26.lib
lapack.lib

Note: because BLAS libraries commonly provide faster versions of some LAPACK subroutines, the BLAS library must be listed before before LAPACK library.
Note: the make sure that the gotoBLAS dll is on your system path e.g. in the WINDOWS\system32 folder, else binary won't run.

7. Compile the project and run the resulting executable. You should get the output:
The solution is -0.661082 9.456125 -16.014625


Prologue:
Part 3 of this HOWTO will briefly explain what "dgesv" means and how to call it and other LAPACK subroutines with the appropriate arguments.

###########################################

Part 3: More about LAPACK subroutines (example: dgesv)

###########################################

(Note: this is the first draft of this section.)

In the previous section, I explained how to call a LAPACK subroutine e.g. dgesv_ from a C or C++ program, but I did not explain what the dgesv meant as well as its arguments. This is the purpose of this part of the HOWTO. In doing so, I will refer to the LAPACK documentation and hopefully show how easy it is to find an appropriate LAPACK subroutine and create the corresponding C/C++ function prototype for it.

Understanding dgesv

Prefix -- "dge"
From the LAPACK naming scheme -- http://www.netlib.org/lapack/lug/node24.html, it is plain to see that:
- d in "dgesv" means: double precsion data
- ge in "dgesv" means: general as in unsymmetric matrix
As such we can deduce that "dge" refers to the sort of matrix we have -- a general/unymmetric matrix containing double precision data

Suffix -- "sv"
This refers to the type of driver routine (solver in lay speak) to be used to solve the linear system. There are two kinds on drivers: simple drivers (suffixed with "sv") and "expert" drivers (suffixed with "svx"). Refer to http://www.netlib.org/lapack/lug/node26.html .

Therefore dgesv is simple driver routine for a general/unymmetric matrix containing double precision data.

Subroutine arguments
From the page http://www.netlib.org/lapack/double/dgesv.f , we can see that the subroutine dgesv has 8 arguments.

- The first argument is N, an integer. This is marked as an input (meaning argument will not be modified, as opposed to an input argument or an input/output argument) in the documentation. In C/C++ speak we can therefore refer to argument 1 as a constant integer i.e. "const int". However, because in Fortran all ALL arguments, *without exception* are passed by address, the type of N in C/C++ is: const *int . (Same goes for argument 2.)

- Argument 3, marked in the documentation as an input/output double precision array. In C/C++ terms input/output means NOT-constant. Therefore, because arguments are passed by reference, the type of argument 3 is: double * .

- Argument 5, marked in the documentation as an output integer array. In C/C++ terms this means the argument is not a const. Therefore argument 5 is of type int* . Same goes for argument 8, even though the argument is not an array (remember, all Fortran arguments are passed by address).

It should now be clear why the C/C++ prototype for dgesv is
Code: Select all
extern "C" void dgesv_( const int * , const int * , double * , const int * , int * , double * , const int * , int * );

Prologue:
A pattern for using directly LAPACK subroutines should now be clear.

- First find the appropriate subroutine from the list of available drivers http://www.netlib.org/lapack/lug/node25.html .
- Look up the driver in the index of routines http://www.netlib.org/lapack/explore-html/
- Create a the appropriate C/C++ prototype for the driver.


###########################################

Part 4: SEND US FEEDBACK!

[b]###########################################
We are working hard to improve the LAPACK/CLAPACK Windows support but it seems that users still have problems.
In case of success or failure, please let us know. We would like to know how we are doing, and how we could further help you.
Your post on the forum will be appreciated.

Enjoy.
graphicsRat & Julie

[Next, using CLAPACK in Visual Studio]
admin
Site Admin
 
Posts: 486
Joined: Wed Dec 08, 2004 7:07 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby mbadawi23 » Mon Sep 14, 2009 3:10 pm

- In the Visual Studio command prompt, change directory to the "exports" subdirectory of GotoBLAS and run the command "make dll". The result should be something like: libgoto_<processor_class>-r<a_number>.lib and libgoto_<processor_class>-r<a_number>.dll . For example: libgoto_banias-r1.26.dll, or libgoto_northwood-r1.26.dll
Don't you need to run "make libs" first?

I get an error if I ran "make dll" without running "make libs":
Code: Select all
make: *** No rule to make target `../libgoto2_prescottp-r1.00.lib', needed by `libgoto2_prescottp-r1.00.dll'.  Stop.


So, after making .lib file using "make libs" I ran "make dll" and I got this error:
Code: Select all
./gensymbol win2k x86 dummy 0 0 > libgoto2_prescottp-r1.00.def
gcc -O2 -Wall -m32 -DF_INTERFACE_GFORT -DSMP_SERVER -DMAX_CPU_NUMBER=2 -DASMNAME =_dllinit -DASMFNAME=_dllinit_ -DNAME=dllinit_ -DCNAME=dllinit -DNO_AFFINITY -DV ERSION="1.00" -I.. -c -o dllinit.obj -s dllinit.c
ranlib ../libgoto2_prescottp-r1.00.lib
dllwrap -mno-cygwin -o libgoto2_prescottp-r1.00.dll --def libgoto2_prescottp-r1.00.def \
        --entry _dllinit@12 -s dllinit.obj --dllname libgoto.dll ../libgoto2_pre
scottp-r1.00.lib -Lc:/mingw/bin/../lib/gcc/i386-pc-mingw32/4.3.3 -Lc:/mingw/bin/
../lib/gcc -Lc:/mingw/i386-pc-mingw32/lib -Lc:/mingw/bin/../lib/gcc/i386-pc-ming
w32/4.3.3/../../../../i386-pc-mingw32/lib -Lc:/mingw/bin/../lib/gcc/i386-pc-ming
w32/4.3.3/../../..  -lgfortran -lgfortran -lmingw32 -lmoldname -lmingwex -lmsvcr
t -lmingw32 -lmoldname -lmingwex -lmsvcrt
dllinit.obj:dllinit.c:(.text+0x22): undefined reference to `blas_memory_alloc'
dllinit.obj:dllinit.c:(.text+0x2d): undefined reference to `blas_cpu_number'
dllinit.obj:dllinit.c:(.text+0x36): undefined reference to `blas_server_avail'
dllinit.obj:dllinit.c:(.text+0x43): undefined reference to `blas_memory_free'
dllinit.obj:dllinit.c:(.text+0x59): undefined reference to `blas_get_cpu_number'

dllinit.obj:dllinit.c:(.text+0x61): undefined reference to `blas_thread_init'
collect2: ld returned 1 exit status
dllwrap: gcc exited with status 1
make: *** [libgoto2_prescottp-r1.00.dll] Error 1


Any help is appreciated. Thank you.
mbadawi23
 
Posts: 11
Joined: Fri Feb 06, 2009 1:16 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby graphicsRat » Tue Sep 22, 2009 8:33 am

mbadawi23 wrote:Don't you need to run "make libs" first?


I don't think so. Did you add the Cygwin path (bin) to your environment variable? (Note that "make dll" should be run NOT by Cygwin but by the visual studio command prompt, and in the exports subfolder). Take a look at the following thread:
viewtopic.php?f=2&t=1593

Please let me know if you have solved the problem. Your experiences would help fine-tune the HOWTO.
graphicsRat
 
Posts: 84
Joined: Wed Mar 25, 2009 3:08 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby flexo1234 » Mon Sep 28, 2009 8:34 am

First of all many thanks for this howto, but I got a few problems.

I tried to compile the gotoblas library following the steps you say, but if I type "make dll" in the command prompt it says it doesn't know the command make.

Also after I did "make" in cygwin, the gotoblas.lib and gotoblas.dll already got compiled and were in the exports folder. Are these the right libraries to use with visual c++ already then? Because if I try them in Visual Studio in debug mode they work, but if I try them in release mode I get errors like "Access violation reading location 0x00905a4d".
flexo1234
 
Posts: 7
Joined: Mon Sep 28, 2009 8:18 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby mbadawi23 » Mon Sep 28, 2009 3:39 pm

but if I type "make dll" in the command prompt it says it doesn't know the command make.
Make sure your environment variables are set right. You want to call "Microsoft nmake" which is what gets called if you are running from a Visual Studio Command Prompt and called "vcvarsall.bat". Here is what the how-to have to say:
4. [Edit:] Now for the final part, building a Windows library (*.lib) and dll for GotoBLAS. This is best done in what's called the "Visual Studio command prompt". The Visual Studio command prompt is basically a version of of the command prompt (cmd) that knows the locations of all the Visual Studio tools e.g. compiler, linker.

That doesn't mean I got my libraries to work :( I'm still struggling.
mbadawi23
 
Posts: 11
Joined: Fri Feb 06, 2009 1:16 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby flexo1234 » Tue Oct 06, 2009 5:52 am

Ok I needed to put project properties->linker->optimization->references to default to let it work in release mode.

But now if I compare the time it takes to do a matrix-vector multiplication with this optimized BLAS library and with Matlab, Matlab is still a bit faster. Is this normal? Did anyone else compare the speed between these?
flexo1234
 
Posts: 7
Joined: Mon Sep 28, 2009 8:18 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby graphicsRat » Mon Oct 12, 2009 8:39 pm

flexo1234 wrote:But now if I compare the time it takes to do a matrix-vector multiplication with this optimized BLAS library and with Matlab, Matlab is still a bit faster. Is this normal? Did anyone else compare the speed between these?


Did you use the BLAS subroutines SGEMV and DGEMV (GotoBLAS implements both). They should outperform Matlab by many orders of magnitude. I would be very surprised if they didn't.
graphicsRat
 
Posts: 84
Joined: Wed Mar 25, 2009 3:08 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby graphicsRat » Mon Oct 12, 2009 8:41 pm

mbadawi23 wrote:
graphicsRat wrote:[Edit:] Now for the final part, building a Windows library (*.lib) and dll for GotoBLAS. This is best done in what's called the "Visual Studio command prompt". The Visual Studio command prompt is basically a version of of the command prompt (cmd) that knows the locations of all the Visual Studio tools e.g. compiler, linker.

That doesn't mean I got my libraries to work :( I'm still struggling.


I would like to help but you need to explain carefully what you done and what's gone/going wrong.

Send me a PM when you post your update.
graphicsRat
 
Posts: 84
Joined: Wed Mar 25, 2009 3:08 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby flexo1234 » Tue Oct 13, 2009 5:10 am

graphicsRat wrote:
flexo1234 wrote:But now if I compare the time it takes to do a matrix-vector multiplication with this optimized BLAS library and with Matlab, Matlab is still a bit faster. Is this normal? Did anyone else compare the speed between these?


Did you use the BLAS subroutines SGEMV and DGEMV (GotoBLAS implements both). They should outperform Matlab by many orders of magnitude. I would be very surprised if they didn't.


Yes I used dgemv. It took almost 7 seconds for a 8000x8000 matrix with a 8000-vector. Matlab does it in 100ms.
If I compare it with the non-optimized BLAS it's only twice as fast (14 seconds), maybe only because gotoBlas is using 2 cpu's.
flexo1234
 
Posts: 7
Joined: Mon Sep 28, 2009 8:18 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby vitaly333 » Mon Oct 19, 2009 12:57 pm

Hello all,
I need to compile gotoBlas under Windows.
I do everything as written in the instruction:
1) install cygwin, make and gnu compilers to C:/cygwin. Set the environment variable to Windows Path C:/cygwin/bin
2) download and extract gotoblas to the C:/cygwin/home/gotoblas
3) cd into gotoblas directory and run :
After some time I get gotoblas library libgoto_athlonp_r1.26.a (and libgoto.a).
4) then i run VS command prompt, cd into C:/cygwin/home/gotoblas/exports and run command make dll:
and I get:
Code: Select all
make: *** No rule to make target '../libgoto_athlonp_r1.06.lib , needed by 'libgoto.dll' . Stop
vitaly333
 
Posts: 16
Joined: Mon Oct 19, 2009 9:57 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby admin » Mon Oct 19, 2009 12:59 pm

Maybe it is just a spelling mistake...
libgoto_athlonp_r1.26.a (

and
libgoto_athlonp_r1.06.lib


You mentioned two different names in your post.
Julie
admin
Site Admin
 
Posts: 486
Joined: Wed Dec 08, 2004 7:07 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby vitaly333 » Mon Oct 19, 2009 1:17 pm

correctly:
Code: Select all
cat: ftest.s: No such file or directory
make: *** No rule to make target `../libgoto_athlonp-r1.06.lib', needed by `lib
goto.dll'.  Stop.

I check, the library libgoto_athlonp-r1.06.lib not exist in the gotoblas directory after compile. There is only a library libgoto_athlonp-r1.06.a.
vitaly333
 
Posts: 16
Joined: Mon Oct 19, 2009 9:57 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby admin » Mon Oct 19, 2009 1:22 pm

Just read the posts above, that problem already occurred for a user.
Did you add the Cygwin path (bin) to your environment variable? (Note that "make dll" should be run NOT by Cygwin but by the visual studio command prompt, and in the exports subfolder). Take a look at the following thread:
viewtopic.php?f=2&t=1593

Please let me know if you have solved the problem. Your experiences would help fine-tune the HOWTO.


Julie
admin
Site Admin
 
Posts: 486
Joined: Wed Dec 08, 2004 7:07 pm

Re: HOWTO LAPACK/CLAPACK for Windows

Postby vitaly333 » Mon Oct 19, 2009 1:43 pm

I have already read it and tried the options suggested there (namely running make dll through the command prompt ). But problems remained
vitaly333
 
Posts: 16
Joined: Mon Oct 19, 2009 9:57 am

Re: HOWTO LAPACK/CLAPACK for Windows

Postby graphicsRat » Mon Oct 19, 2009 4:42 pm

vitaly333 wrote:
Code: Select all
cat: ftest.s: No such file or directory
make: *** No rule to make target `../libgoto_athlonp-r1.06.lib', needed by `lib
goto.dll'.  Stop.

I check, the library libgoto_athlonp-r1.06.lib not exist in the gotoblas directory after compile. There is only a library libgoto_athlonp-r1.06.a.


As far as I know, only the *.a file is required. The *.lib and *.dll files are generated in the next step, using the visual studio command prompt. For example:

Code: Select all
C:\>cd Downloads\src\GotoBLAS-1.26\GotoBLAS\exports

C:\Downloads\src\GotoBLAS-1.26\GotoBLAS\exports>make dll
./gensymbol win2k dummy  0 X86 > libgoto_banias-r1.26.def
gcc -O2 -D_GNU_SOURCE -DWINDOWS_ABI -Wall -m32 -DF_INTERFACE_F2C -DNEED_F2CCONV -DMAX_CPU_NUMBER=2 -DNUM_BUFFERS=\(2*2\) -DASMNAME=_dllinit -DASMFNAME=_dllinit_ -DNAME=dllinit_ -DCNAME=dllinit -DFUNDE
RSCORE=_ -DNEEDFUNDERSCORE -DBUNDERSCORE=_ -DNEEDBUNDERSCORE -I.. -DARCH_X86 -DPENTIUMM -DL1_CODE_SIZE=32768 -DL1_CODE_ASSOCIATIVE=8 -DL1_CODE_LINESIZE=64 -DL1_DATA_SIZE=32768 -DL1_DATA_ASSOCIATIVE=8
-DL1_DATA_LINESIZE=64 -DL2_SIZE=1048576 -DL2_ASSOCIATIVE=8 -DL2_LINESIZE=64 -DITB_SIZE=4096 -DITB_ASSOCIATIVE=4 -DITB_ENTRIES=128 -DDTB_SIZE=4096 -DDTB_ASSOCIATIVE=4 -DDTB_ENTRIES=128 -DHAVE_CMOV -DHA
VE_MMX -DHAVE_SSE -DHAVE_SSE2 -DHAVE_CFLUSH -DNUM_SHAREDCACHE=1 -DNUM_CORES=1 -DCORE_BANIAS  -c -o dllinit.o -s dllinit.S
dllwrap -mno-cygwin -o libgoto_banias-r1.26.dll --def libgoto_banias-r1.26.def \
        --entry _dllinit@12 -s \
        dllinit.o ../libgoto_banias-r1.26.a
lib /machine:i386 /def:libgoto_banias-r1.26.def
Microsoft (R) Library Manager Version 7.10.6030
Copyright (C) Microsoft Corporation.  All rights reserved.

   Creating library libgoto_banias-r1.26.lib and object libgoto_banias-r1.26.exp

C:\Downloads\src\GotoBLAS-1.26\GotoBLAS\exports>


This works on both machines that I use. If you're still having problems compiling the dll, consider emailing Kazushige Goto (author of GotoBLAS) at kgoto[at]tacc[dot]utexas[dot]edu. I'm sure he'll help if you ask nicely.
graphicsRat
 
Posts: 84
Joined: Wed Mar 25, 2009 3:08 pm

Next

Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 2 guests