Extracting data from a VNL file

[ipsecog]

Putting data into VNL files is simple in NanoLanguage, using the addToSample() function, and appears to be a convenient way of storing data calculated by ATK. However, getting it out again can be a challenge (which kind of makes the idea of using it for storing data a bit less useful). Of course, Virtual NanoLab can read the data and plot it, but what if you want the actual data, the raw numbers? That is, the data you had access to before you put it into the VNL file (typically via the toArray() method on the data object, e.g. the transmission spectrum). Well, it turns out that the vnl files have a rather simple but undocumented structure; they are actually just Python zipfiles, and the data is stored in nested dicts with quite obvious labels! Therefore, it's relatively simple to extract the data, as long as you know what you are doing. Disclaimer: The scripts below use undocumented features, and may not work in future (and/or some old) releases of ATK. It's best to do some of these operations in interactive mode, but there is an initial part, which is in the attached script. Download it and run it as

Code

atk -i extract_from_vnl_file.py

Note the "-i" option; this will cause ATK to execute the script and then remain in interactive mode. The script will ask for the filename (because it cannot be given as a script parameter when you want to run in interactive mode); be sure to specify it with full path if it's not in the current directory. (On most platforms you can use the Tab key to complete the file name!) At this point the script has created a variable (a dictionary) called samples which contains all the data. In this top-level dict, the keys correspond to the sample names. You can list the sample names:

Code

print samples.keys()

Let us assume that the sample we are interested in is called "mySample". To drill down to the data, give the command

Code

data = samples['mySample']['NanoLanguage']

Now, you can see which data has been stored on this sample:

Code

print data.keys()

Let us assume that we are looking to extract a transmission spectrum. In that case, you will hopefully see (perhaps among many other things) a key called "Transmission Spectrum Transmission Spectrum" in the printed list. However, it depends a bit on the version of ATK you used to create the VNL file. In some older versions it's just "Transmission Spectrum" and in some really old ones it might even be "TransmissionSpectrum". So, it's best to inspect the data to be sure! Now we can extract the data:

Code

trans = data["Transmission Spectrum Transmission Spectrum"]

By typing "trans" and hitting the Tab key, we now find that the so-created "trans" data object has several methods and properties. (You can also give the command

Code

dir(trans)

to see the list. What we are interested in here are the energies E and the values of the transmission at these energies, T(E). Here's how to access it:

Code

E = t.energy()._dataarray_
T = t.average_transmission()

And now we can just print the transmission spectrum (or do whatever you want with it):

Code

for i in range(len(E)):
    print E[i],T[i]

Note that if the calculation was spin-polarized, the transmission has a bit different shape. Using the general method outlined above it should be possible to extract any kind of data stored in a VNL file. Do note, however, that each specific data type is stored in different ways, and you have to figure out the (undocumented) methods and properties for each one by using the "dir()" command quite a lot... To summarize, here is a complete script that extracts and prints the transmission spectrum from a VNL file, by asking the user for the file and sample name (this script is also attached to this post as extract_transmission_from_vnl_file.py):

Code

import zipfile, cPickle, sys

filename = raw_input('Please enter VNL file name: ')
try:
    f = zipfile.ZipFile(filename,'r')
except:
    print 'Unable to locate VNL file "%s"' % filename
    sys.exit(1)

# Restore the samples from the VNL file into a dict
samples = {}
for zinfo in f.infolist():
    s = f.read(zinfo.filename)
    try: 
        obj = cPickle.loads(s)
        samples[obj.name()] = {}
        for h in obj.history():
            samples[obj.name()][h.name] = h.resultsample.properties()
    except:
        pass

print 'The following samples are present in this VNL file:'
print samples.keys()

sname = raw_input('Please enter sample name to extract (default=first sample): ')
if sname.strip()=="":
    data = samples[samples.keys()[0]]['NanoLanguage']
else:
    data = samples[sname]['NanoLanguage']

# Some older versions used "Transmission Spectrum" or "TransmissionSpectrum"
trans = data['Transmission Spectrum Transmission Spectrum']
print 'Energy (eV)\tTotal transmission'
print '----------------------------------'
energies = trans.energy()._dataarray_
# Check for energy unit, if you want to be sure
# e_unit = t.energy().units()
T = trans.average_transmission()
for i in range(len(energies)):
    print '%g\t%g' % (energies[i],T[i])

[Anders Blom]

I have written some further utilities for extracting an effective potential from a VNL file. It can easily be modified for densities too, if someone is interested in this. The typical scenario is that you calculate the e.g. effective potentials together with the SCF loop, and store them in VNL files (perhaps different ones, for each bias). Afterwards, we want to compute e.g. the voltage drop without redoing the effective potentials, which can take a long time. So, we want to extract them from the VNL files, take the difference, and store that back into the VNL file. Now, first of all, it might be hard to remember which sample name was used, and how the objects are actually called inside the VNL files. So, as our first step, we run the (attached) script inspect_vnl_file.py to find out exactly which objects are in the VNL files. When you run the script, it will ask you for the name of the VNL file; you can also give it directly on the command line:

Code

atk inspect_vnl_file.py myfile.vnl

The script will print some output telling us about which things are stored in the VNL file. When we know this, we go over to voltage_drop.py and insert therelevant names. As you see, the two potentials can come from different files, or the same. You can keep

Code

samplename=""

if there is only one sample in the file, or you are just interested in the first sample. For object, either use the number or object name as reported in the output by the "inspection" script. As you see, furthermore in the script, we compute the difference = the voltage drop and save it in a VNL file. Always keep in mind the hierarchy:

• A VNL file can contain several samples; these all have different sample names.
• On each sample you can store several objects.

The sample name is always defined by the user, although many tend to use the default from VNL, like "twoprobe_configuration" for transport systems. Note that writing a sample with the same sample name as an existing one to a file, will overwrite that sample in the file. Formally the objects are referred to by name, given in the script which saved them to the VNL file, or autogenerated by ATK. The script above adds a twist to it, and allows you to refer to them by sequence number. That's also how it works in ATK 2010.xx, where you can choose to read by object type, object id, or object label. In general, we should point out that ATK 2010.xx works much more symmetric than 2008.10. There is always an equivalent command to read things from the NetCDF file, corresponding to the one that writes things to it.

[jdgayles16]

Hi Try this script and I get the error

File "inspect_vnl_file.py", line 2, in ?
    from VNLFileExplorer import *
ImportError: No module named VNLFileExplorer


any solutions?

[jdgayles16]

Sorry silly mistake, I figured it out

[jdgayles16]

Another error,

exceptions.KeyError                                  Traceback (most recent call last)

/home/users3/cmt/jgayles/vnlstuff/<console>

KeyError: 'NanoLanguage'


anyone know the solution?

[Anders Blom]

This would be coming from line 28 or 30 in the VNLFileExplorer. You may need some debug lines, like

Code: python

print samples.keys()
print dir(samples[samples.keys()[0]])

after current line 26. The idea is that "samples" is a dictionary (dict) of all samples in the VNL file. If no samplename is provided, grab the first. However, if that goes well, the object thus returned should always have a dictionary in itself called "NanoLanguage". If it doesn't, I'm guessing the first thing in the VNL file might be just a configuration, without any calculated results. Open the VNL file in the Result Browser to find out the tree structure and sample names, then try again and this time specify the samplename, like

Code: python

read_potential_from_vnl_file("myfile.vnl",samplename="something")

[gump]

When I tried ipsecog's method using ATK2008.10 in the linux system by excuting the extract_from_vnl_file.py file with the command "atk -i extract_from_vnl_file.py", there was an error:
      2
----> 3 filename = raw_input('Please enter VNL file name: ')
      4 try:
      5     f = zipfile.ZipFile(filename,'r')
      6 except:

EOFError: EOF when reading a line
WARNING: Failure executing file: <extract_from_vnl_file.py>
what is the solution? Thanks

[Anders Blom]

As far as I can see this would occur if you run the script, but instead of actually entering the name of the VNL file at the prompt, you press Ctrl+D.

When the prompt appears ("Please enter VNL file name:"), type the name of the VNL file and press Enter.

[Ben]

Thanks for share.
I want to know how to get the DOS data from the vnl file，can you teach me?

[Ben]

Hello,since for transmission spectrum :"T = t.average_transmission()", what is it for DOS?I already know the keys by using the inspect_vnl_file.py,while I still don't know the attribute names.

[Anders Blom]

We can't offer support for this ATK version anymore. You can learn about the methods on a object (say, t) by doing "print dir(t)".

[Ben]

Thank you very much!
I had my problem solved by your means.