Jon’s Project Blog

I developed many finite elements during my Ph.D. research. Many finite element programs are built upon a mix of C/C++/Fortran. I had felt more comfortable with C/C++ over Fortran, and I have become fairly familiar with the C/Fortran calling conventions for struct/common blocks and etc.

Now that I’ve graduated, I no longer have access to Matlab for rapid prototyping and rapid checking of solutions to differential equations that I may be interested in. I recently ran across Julia which is a newer language. I decided to give it a try as I’ve done with R, Octave, and a slew of other “Matlab” clones. Julia has some nice features such as parallelism that I have not touched. It’s definitely not a Matlab clone, but it shares many similarities, and has some degree of built-in parallelism.

Recently, I’ve been involved in another material modeling project, and I’ve decided to use Julia during this project as a “rapid prototyping” replacement for Matlab. I found a gnuplot interface that seems to work pretty well called Gaston that seems to work pretty well. However, ultimately my code must run in Fortran for this project where it is compiled into the finite element code as a user material routine. Therefore, I decided to try to use Julia’s C/Fortran calling interface.

To do this in Julia, you must compile your subroutines and functions into a shared library. To illustrate this, we will look at the following fortran code that will help us investigate the underlying structure of arrays, matrices, and 3rd-order tensors in Julia.

c matrix test file
 
	subroutine vec(n,vector)
 
	integer n
	real*8 vector(*)
	write(*,*) (vector(i), i=1,n)
 
	end
 
	subroutine mat(m,n, matrix)
 
	integer m,n
	real*8 matrix(m,n)
	write(*,*) ((matrix(i,j),j=1,n),i=1,m)
 
	end
 
	subroutine tten(m,n,o,matrix)
 
	integer m,n,o
	real*8 matrix(m,n,o)
	write(*,*) (((matrix(i,j,k),j=1,n),i=1,m),k=1,o)
 
	end

We can then compile the fortran code into a shared object by the following:

gfortran matrix.f -o matrix.o -shared -fPIC

Then in Julia we can paste in the following code:

matrix = dlopen("matrix.o")
vec_ = dlsym(matrix,:vec_)
mat_ = dlsym(matrix,:mat_)
tten_ = dlsym(matrix,:tten_)

a = linspace(1.,4.,4)
ccall(vec_,Void,(Ptr{Int32},Ptr{Float64}),&int32(4),a)
a = linspace(1.,4.,4)'
ccall(vec_,Void,(Ptr{Int32},Ptr{Float64}),&int32(4),a)

b = ones(Float64,4,1)
c = b * a
ccall(mat_,Void,(Ptr{Int32},Ptr{Int32},Ptr{Float64}),&int32(4),&int32(4),c)

d = ones(Float64,4,4,2)
d[:,:,1] = c
ccall(tten_,Void,(Ptr{Int32},Ptr{Int32},Ptr{Int32},Ptr{Float64}),&int32(4),&int32(4),&int32(2),d)

The conclusion is that the underlying representation in Julia matches the ordering you would expect in Fortran for arrays, matrices, and 3rd-order tensors. With a bit of syntactic sugar, one can mask all the pointer casting, and create faster C/Fortran versions of the same Julia code. Hopefully, this languages provides a good way of rapid prototyping codes in a higher-level language, and then porting pieces to more traditional compiled object codes into high performance computing frameworks.

Julia isn’t the easiest language to use at the moment, but it seems to hold much promise. I had to write my ODE solver, which I guess is not too difficult. Now that I have this C/Fortran interfacing figured out, it should be trivial to add in ODE support from libraries from the national labs that have been well tested.

Recently I needed to convert old xfig files to inkscape while preparing figures for my dissertation. Luckily I found out about dktools. However, there wasn’t a convenient package in an ubuntu repository. Fortunately, it’s fairly easy to compile. Just in case someone finds this useful, I’ll leave installation instructions here for Ubuntu users:


sudo apt-get install libpng-dev libdb5.1-dev libbz2-dev libjpeg-dev libtiff-dev libsnmp-dev
./configure
make
make install


To convert a .fig to .svg:

fig2vect -lsvg fig.fig fig.svg


So I initially created a mercurial repository to house my research related publications, presentations, and thesis. However, as these publications generally have coauthors, sharing parts of my repository was necessary. However, I did not want to run two repositories, and would ideally want to keep my current repository structure as it makes sense for me. On the other hand, I do not want to share my whole publication repository with all my coauthors, when what they are really interested in, is the particular publication we are working on. I do however want my collaborators to view my ‘hg log’ history. Fortunately, mercurial (hg) actually can account for this through use of subrepositories. However there is a bit of tricky in setting things up.

1. First, I generated a separate mercurial repository using the current repository, but only for the desired subfolder. The following is a bash shell script that handles moving the logs and desired files over. The first argument is the subfolder you want to make into a repository. The second argument is the path to the parent directory. The third argument is the desired location of the new repository.

echo include $1 > /tmp/myfilemap
echo rename $1 . >> /tmp/myfilemap
hg convert --filemap /tmp/myfilemap $2 $3

2. Next the subfolder is removed first from the parent repoistory via ‘hg remove path/to/subfolder’.
3. Following the instructions from the mercurial wiki, one goes to the root directory of the parent repository and create the necessary .hgsub file. In my case, my subfolder was 2 levels down, so the following was used. The paths are relative the the root of the parent directory.

folder/subfolder = folder/subfolder

4. ‘hg add .hgsub’ in the parent directory.
5. ‘hg clone subfolder’ in the folder/subfolder.
6. ‘hg commit -m “description here”‘

At this point, it seems everything works as desired. Pulling, updating, and cloning the root directory records the changes of the subfolder, while the subfolder can itself be updated/pulled/and pushed in accordance with collaborators.

I reviewed paraview about two years ago, and I more or less lambasted it for not being very well documented and that the mailing list was not as responsive to “beginners” questions as one might imagine. Some of the examples on the wiki also did not work back then. I have been quietly monitoring and occasionally evaluating paraview to see how it has been improving. In one of the recent versions this year, they actually released the user guide/handbook, which contains some useful information. The file reader formats have greatly improved and a lot of the features rival that of tecplot, which is not free.

While my research lab recently acquired Abaqus, all of my dissertation code is in the Finite Element Analysis Program (FEAP) from Berkeley. It was written mainly by Robert Taylor (one of the Godfathers of finite elements) and by Sanjay Govindjee. It is a pretty good code, and is written mainly in Fortran/C. Unfortunately, while it includes some sophisticated numerical schemes and efficient solvers, it does not provide much in terms of pre- and post-processing for finite elements. In the past we used Ansys to generate meshes, and manipulated the boundary conditions by hand. However, we have since then, obtained patient-specific heart meshes, and the number of nodes is much larger and the geometry/numbering is not so obvious.

Fortunately, paraview has some powerful selection filters and tools available. Unfortunately, after each selection the number of elements and nodes is renumbered. Paraview actually keeps track of the global id’s but for some reason it is filtered out from the spreadsheet view by default. However this can be changed easily. To use paraview as a pre-processor, the idea is to use successive “Extract Selection” filters after using the “Fustrum” or “Surface” selection tool for nodes/elements in the Selection Toolbar. One should think of it as taking the original mesh, and then slicing or cutting away the parts that are not of interest until you are left with only what you want. Lastly, in the selection explorer, you can also select “Invert Selection”, which effectively allows you to select the portion you want to “cut-away” from the mesh.

The following are instructions to perform this pre-processing selection using paraview.

1. If not already done, go to Preferences > Charts. Then delete the line that says “vtkOriginalIds” that is in the “hidden” list. This will show the “vtkOriginalIds” value in the spreadsheet view and allow you to “Save Data” on each “Extract Selection”, such that one will get a mapping from vtkOriginalIds to the renumbered fields.
2. Use the provided selection tools, and “Extract Selection” successively until you arrive at the selection you want.
3. Click on each “Extract Selection” filter and click on File > Save Data. Select a filename, and make sure to specify either Point/Cell/Field data so you save the proper mapping.
4. Next you can use the following python script to get the proper mapping using 0-index numbering (paraview indexing). The script takes in a list of “Extraction Selection” csv files and an outputfilename for the global id mapping.

import os,sys
 
def parse(args):
    extractions, outputfile = map(open,args[:-1]),open(args[-1],"w")
 
    ids = []
    newids = []
 
    for extraction in extractions:
        origids = []
        for line in (extraction.readlines())[1:]:
            orig= line.split(",")[0]
            origids.append(int(orig))
 
        # if ids is filled map rewrite ids accor
        if len(ids) > 0:
           for id in range(len(origids)):
               origids[id] = ids[origids[id]]
 
        ids = origids
 
    for id in ids:
        outputfile.write("%d\n" % (id))
 
    outputfile.close()      
 
if __name__=="__main__":
    parse(sys.argv[1:])