# JULES NEON suite **URL:** \ **Category:** JULES\ **Created:** [29 September 2022 11:54 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734 "2022-09-29T11:54:02Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![tquaife](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@tquaife](https://cms-helpdesk.ncas.ac.uk/u/tquaife)\ **Post date:** [29 September 2022 11:54 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/1 "2022-09-29T11:54:02Z")
Hi, I’d like to have a a suite to run JULES at NEON terrestrial sites in the US. Observed meteorology should be available via the NEON API, but could use reanalysis as an alternative. Thanks, Tristan. ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 12:07 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/2 "2022-09-29T12:07:07Z")
Thanks, Tristan: I have started working on this, using the u-al752 JULES FLUXNET suite as a basis. We have contacted the primary owners of u-al752 (Karina Williams and Anna Harper), and they have pointed me to some code that might be able to be adapted for adding the JULES NEON sites to the u-al752 suite. I also tried to download some of the NEON data from their website. It is a bit slow to download. Maybe it’s faster (maybe there isn’t any data throttling) if I get a NEON account? I will try that. Patrick ---
**Author:** ![tquaife](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@tquaife](https://cms-helpdesk.ncas.ac.uk/u/tquaife)\ **Post date:** [29 September 2022 12:46 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/3 "2022-09-29T12:46:45Z")
I don’t know if having an account will speed up the download - I suspect not, but worth a try. ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 13:32 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/4 "2022-09-29T13:32:52Z")
Hi Tristan: I thought I saw somewhere on the NEON site that there might be throttling or something, for users without a NEON account. But I don’t immediately see that right now. Anyways, I will try downloading after creating an account soon. Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 18:31 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/5 "2022-09-29T18:31:46Z")
Hi Karina and Anna Tristan suggests making a suite that can run JULES for the NEON terrestrial sites (approx. 30 sites). When I heard this idea, I immediately thought of the u-al752 JULES FLUXNET suite that I worked with you to get working on JASMIN. I suggested to Tristan that maybe I could adapt that suite to work with the NEON sites. Tristan doesn’t think that it is absolutely necessary to have the capability within the suite to overplot the time series of the observations from the towers on the same plot as the time series of the JULES model outputs, but I think that is interesting and useful to work towards porting that capability from your suite to the NEON-capable version of your suite. Would it be OK to adapt your u-al752 suite so that it can run JULES for the NEON sites? Do you know of anybody else who has already done something like this? You do have some tools that are not part of the u-al752 JULES FLUXNET suite for preprocessing the atmospheric-forcing data and populating the soil/vegetation/etc. ancillary data fields. Do you think those tools would be useful for the NEON sites? Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 18:34 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/6 "2022-09-29T18:34:10Z")
Hi Patrick, Sure, you can use u-al752 - you could add the NEON sites directly to the u-al752 trunk if you like? I’ll forward the info summary of how to put new FLUXNET sites into the suite, as lots of that can be adapted I think. > You do have some tools that are not part of the u-al752 JULES FLUXNET suite for preprocessing > the atmospheric-forcing data and populating the soil/vegetation/etc. ancillary data fields. Do you > think those tools would be useful for the NEON sites? Yes, I think so. The code is stored at [https://code.metoffice.gov.uk/trac/utils/wiki/smstress\_jpeg](https://code.metoffice.gov.uk/trac/utils/wiki/smstress_jpeg) [https://code.metoffice.gov.uk/trac/roses-u/browser/a/l/7/5/2/u-al752-processing/bin](https://code.metoffice.gov.uk/trac/roses-u/browser/a/l/7/5/2/u-al752-processing/bin) All of this code works on python 2.7 (if you need a python2.7 environment to run them, you can use the MONSOON postprocessor). Some of the scripts work in both python 2.7 and python 3 (using **future** imports, e.g. [https://code.metoffice.gov.uk/trac/utils/browser/smstress\_jpeg/trunk/jules.py#L18](https://code.metoffice.gov.uk/trac/utils/browser/smstress_jpeg/trunk/jules.py#L18)). If you fancy adapting some of the other scripts to also run on both python 2.7 and 3, that would give us a lot more flexibility for running the suite, especially [https://code.metoffice.gov.uk/trac/utils/browser/smstress\_jpeg/trunk/fluxnet\_evaluation.py](https://code.metoffice.gov.uk/trac/utils/browser/smstress_jpeg/trunk/fluxnet_evaluation.py) and [https://code.metoffice.gov.uk/trac/utils/browser/smstress\_jpeg/trunk/test\_fluxnet\_evaluation.py](https://code.metoffice.gov.uk/trac/utils/browser/smstress_jpeg/trunk/test_fluxnet_evaluation.py) (I’ve been using this script for lots of non-FLUXNET sites, so it should hopefully be useful for NEON sites too) Cheers, Karina ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 18:34 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/7 "2022-09-29T18:34:52Z")
Hi Patrick That sounds great, I don’t know of other work to add NEON sites to the suite. Thanks Karina for sending along the scripts! Cheers, Anna Harper ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 18:43 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/8 "2022-09-29T18:43:28Z")
Hi Fred [and Patrick]: [originally sent by email to Fred Otu-Larbi, CC: Patrick McGuire et al., in July 2020] I’ve added site BR-Sa3 to u-al752 and [in the next post in this 2022 ticket, I attach] a description of how I did it (notes\_on\_adding\_a\_site.txt), which hopefully you can follow for the sites you need: US-Blo and US-Sa1 [or the NEON sites]. But if any steps don’t make sense, give me an email and we can work through them together over skype. I’m sure there’ll be things I forgot to write down, so there’s anything not immediately clear from the description, email me. These are the changes made in u-al752: [https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164463%40a%2Fl%2F7%2F5%2F2%2Ftrunk&old=164432%40a%2Fl%2F7%2F5%2F2%2Ftrunk](https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164463%40a%2Fl%2F7%2F5%2F2%2Ftrunk&old=164432%40a%2Fl%2F7%2F5%2F2%2Ftrunk) and these are the changes made in u-al752-processing: [https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164464%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing&old=164428%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing](https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164464%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing&old=164428%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing) [I’ve sent plots by email] showing LBA-K83 and BR-Sa3. It doesn’t include any processing of soil moisture or LAI at the moment - that’ll be the next stage. Next, I’ll plot the soil moisture observations from FLUXNET2015, Ameriflux and [LBA-Ecology Km 83 Data, Publications, and Presentations](https://www.ess.uci.edu/~lba) for this site, and see how they compare to the observations for this site from LBA-MIP. cheers, Karina ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [29 September 2022 18:47 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/9 "2022-09-29T18:47:42Z")
**notes\_on\_adding\_a\_site.txt** Karina Williams 6.7.2020 **Adding a new FLUXNET2015 site BR-Sa3 (without prescribing soil moisture or LAI)** **Changes** These are the changes made in u-al752:[https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164463%40a%2Fl%2F7%2F5%2F2%2Ftrunk&old=164432%40a%2Fl%2F7%2F5%2F2%2Ftrunk](https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164463%40a%2Fl%2F7%2F5%2F2%2Ftrunk&old=164432%40a%2Fl%2F7%2F5%2F2%2Ftrunk) and these are the changes made in u-al752-processing:[https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164464%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing&old=164428%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing](https://code.metoffice.gov.uk/trac/roses-u/changeset?reponame=&new=164464%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing&old=164428%40a%2Fl%2F7%2F5%2F2%2Fu-al752-processing) **Steps** 1. Download Tier 1+2 FLUXNET2015 data for Br-Sa3. Unzip. Check out trunk and processing branches if you don’t have them already: cd ~/roses svn co [https://code.metoffice.gov.uk/svn/roses-u/a/l/7/5/2/trunk](https://code.metoffice.gov.uk/svn/roses-u/a/l/7/5/2/trunk) u-al752 svn co [https://code.metoffice.gov.uk/svn/roses-u/a/l/7/5/2/u-al752-processing](https://code.metoffice.gov.uk/svn/roses-u/a/l/7/5/2/u-al752-processing) u-al752-processing If you have them already, update them (“svn update”) to check you have the latest version. 2. Create a file called filelist\_br-sa3.txt out of an edited version of the ‘download manifest’ text that appears on the screen that gives the links to download. Because only one more site is being added, this file just contains the line: `BR-Sa3,2020-06-30/FLX_BR-Sa3_FLUXNET2015_FULLSET_HH_2000-2004_1-4.csv,2000,2004,1-4,FLUX-MET,202001230043` Put the name of this file in u-al752-processing/bin/convert\_for\_jules\_edited\_for\_suite\_u-al752.py (see the code comments in the main function for more information) and run this script. This produces files in u-al752-processing/processed\_files/drive and u-al752-processing/processed\_files/fluxes/subdaily\_obs 3. Change the site list in u-al752-processing/bin/create\_daily\_GMT\_files.py and run. This produces files in u-al752-processing/processed\_files/fluxes 4. Create a new folder vn1.2 within your suite\_data folder. Copy across the files from vn1.1. Copy across the new file u-al752-processing/processed\_files/drive/BR\_Sa3-met.dat and the new file u-al752-processing/processed\_files/fluxes/BR\_Sa3-energyandcarbon-dailyUTC.dat 5. Add sites to the site\_info\_fluxnet dictionary in the get\_site\_info function in u-al752-processing/bin/generate\_conf\_files.py. Specify the type if you don’t want to use the category given in u-al752-processing/bin/fluxnet2015\_site\_info.csv. The file with this type in the filename must exist in u-al752/ancil/tilefracs (if it’s not there already, then create it). Years are the same as in the original ‘download manifest’. HH is timestep 1800 seconds and HR is timestep 3600 seconds. i.e. add “BR\_Sa3” : {“drive\_startyear” : 2000, “drive\_endyear” : 2004, “type” : “BET-Tr”, “timestep” : 1800, }, 6. If have site fractions of sand/silt/clay, add to u-al752-processing/bin/soil\_texture.dat If have other site obs for soil e.g. wilting point and field capacity, add this in explicitly to the make\_soil\_props\_files function in u-al752-processing/bin/create\_presc\_soil\_moisture.py (following the methods used for AT\_Neu, BE\_Vie, FI\_Hyy). If have used site sand/silt/clay or other data, add this information in to the function print\_soil\_properties in u-al752-processing/bin/generate\_soil\_props.py (e.g. see notes\_dict[‘AT-Neu’]) Make links to files qrdata.soil.texture and HadGEM2ES\_Soil\_Ancil.nc, and the folder suite\_data in the folder u-al752-processing/links 7. Add new site to site\_info in u-al752/var/info.inc. Change DATA\_VERSION in u-al752/suite.rc to 1.2. 8. Make sure python has access to the scripts listed in u-al752-processing/bin/README. Run u-al752-processing/bin/generate\_soil\_props.py (will generate the files of soil properties in u-al752/ancil/soil and the ancil conf file u-al752/app/jules/opt/rose-app-ancil-BR\_Sa3.conf) Compile u-al752-processing/bin/display\_soil\_props\_table.tex (i.e. “pdflatex display\_soil\_props\_table.tex”) and check that the resulting file u-al752-processing/bin/display\_soil\_props\_table.pdf looks ok. 9. Create a link within suite\_data called link-to-current-version that points to vn1.2. Run u-al752-processing/bin/generate\_conf\_files.py. Will create the drive conf file u-al752/app/jules/opt/rose-app-drive-BR\_Sa3.conf. Check there’s no messages saying file is unexpected length (i.e. no warning to say the driving data does not have the expected amount of data). 10. “svn add” all the new files in u-al752. Run the suite to check it works ok with the new sites. Compare new plots to the plots made by the last version. Commit changes to u-al75 and u-al752-processing folders. Copy the vn1.2 data to MONSooN. Make tarball of vn1.2 data (change version number, SUITE\_DIR and TMP\_DIR in u-al752-processing/bin/archive\_data.bat and run) and put on dropbox. N.b. the tarball on dropbox shouldn’t include the soil moisture and LAI files until we have sorted out what can be used beyond the original study (i.e. Anna’s paper) and provided guidelines on how the data should be credited. Update table on twiki page ([https://code.metoffice.gov.uk/trac/jules/wiki/FluxnetandLbaSites](https://code.metoffice.gov.uk/trac/jules/wiki/FluxnetandLbaSites)) and the “processed obs files” section. Attach plots and display\_soil\_props\_table.pdf. Add any particular information about the site to the “Site specific information” section. Email rest of u-al752 team. N.b. some of the sites are also in the LBA and/or Ameriflux networks (BR-Sa3 is in both) - so these are other places to download data from. There could also be extra infomation on site websites e.g. BR\_Sa3 has data at [LBA-Ecology Km 83 Data, Publications, and Presentations](https://www.ess.uci.edu/~lba/). ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [10 October 2022 15:10 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/10 "2022-10-10T15:10:08Z")
Hi Tristan I have looked at the temperature data for the NEON SCBI site. I am wondering what heights the 5 vertical levels are at. Do you know where I can find this information? I did all sorts of Google searches for it, and I can’t see the vertical level heights in the data files. More generally, a table of the vertical levels for the various sites would be useful to me, if it’s not in each site’s dataset somewhere already. ``` I have one month of data here in the landsurf_rdg GWS on JASMIN, if you'd like to look at it. You can apply for access to that GWS first. This is the path for the 1 month of NEON data: ``` /gws/nopw/j04/landsurf\_rdg/pmcguire/TristanQ/NEON.D02.SCBI.DP4.00200.001.2017-01.basic.20220120T173946Z.RELEASE-2022 Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [10 October 2022 15:10 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/11 "2022-10-10T15:10:35Z")
Hi Patrick, Which data product is it? Taking DP1.00002.001 (Single aspirated air temperature) as an example, then you can check the data endpoint of the API: [https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02](https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02) (Note - my choice of month here was arbitrary). That endpoint just returns the URL of a bunch of files, and one of them is: NEON.D02.SCBI.DP1.00002.001.sensor\_positions.20220313T193816Z.csv And that contains the sensor heights. Full URL is: [https://storage.googleapis.com/neon-publication/NEON.DOM.SITE.DP1.00002.001/SCBI/20220201T000000--20220301T000000/expanded/NEON.D02.SCBI.DP1.00002.001.sensor\_positions.20220313T193816Z.csv](https://storage.googleapis.com/neon-publication/NEON.DOM.SITE.DP1.00002.001/SCBI/20220201T000000--20220301T000000/expanded/NEON.D02.SCBI.DP1.00002.001.sensor_positions.20220313T193816Z.csv) I assume there will be similar files for other data types too. What I don’t fully understand is why the heights are described as an “offset” … offset from what? Presumably the ground, but I am not sure. Cheers, Tristan. ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [10 October 2022 15:11 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/12 "2022-10-10T15:11:06Z")
Hi Tristan That’s impressive that you found that info with little trouble. I had downloaded with the browser one month of data from “Bundled data products - eddy covariance” in the landsurf\_rdg GWS on JASMIN. /gws/nopw/j04/landsurf\_rdg/pmcguire/TristanQ/NEON.D02.SCBI.DP4.00200.001.2017-01.basic.20220120T173946Z.RELEASE-2022 The weblink that has this data is, for example: [https://data.neonscience.org/data-products/DP4.00200.001](https://data.neonscience.org/data-products/DP4.00200.001) But I can’t figure out which buttons to push to also download endpoint link for the sensor\_positions file that you found in the API directory [https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02](https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02) Can you tell me how you found that API endpoint link? I have been starting out with [https://data.neonscience.org/data-products/explore](https://data.neonscience.org/data-products/explore) Patrick ---
**Author:** ![tquaife](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@tquaife](https://cms-helpdesk.ncas.ac.uk/u/tquaife)\ **Post date:** [10 October 2022 15:35 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/13 "2022-10-10T15:35:09Z")
Hi, I normally start with the site endpoint, e.g. for SCBI: [https://data.neonscience.org/api/v0/sites/SCBI](https://data.neonscience.org/api/v0/sites/SCBI) and that lists all the products for that site and the dates they are available. Then I use the data endpoint to examine the contents what data is available for that, e.g.: [https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02](https://data.neonscience.org/api/v0/data/DP1.00002.001/SCBI/2022-02) There’s normally a bunch of meta-data included at that level, but it can be a pain to spot. Is it possible this is why downloads were slow for you? If you use the data endpoint it gives you the address of individual csv files so you can fine tune what you want. Cheers, Tristan. ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [10 October 2022 19:15 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/14 "2022-10-10T19:15:40Z")
Thanks, Tristan: I understand the site endpoint better now ([https://data.neonscience.org/api/v0/sites/SCBI](https://data.neonscience.org/api/v0/sites/SCBI)). I see the various data products for the SCBI Smithsonian NEON site, including DP4.00200.001 (Bundled data products - eddy covariance). And I now know how to find out sensor positions for unbundled products. I am now downloading almost 6 years of DP4.00200.001 bundled data for SCBI from the [https://data.neonscience.org/data-products/explore](https://data.neonscience.org/data-products/explore) website. This amounts to 5.5GB, and it is being downloaded at almost 4 MB/s, which is much faster than before. Previously, I attempted to download a dataset from there, and it was going at only 0.6-0.7 MB/s, so I gave up. For the 5.5GB, it is more than half done now, and it’s only been of order 10 minutes or so. Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [12 October 2022 22:38 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/15 "2022-10-12T22:38:03Z")
Hi Tristan: Anecdotal trials of the downloads from the NEON site suggest that it can maybe be 2x faster downloading when downloading with a NEON account, versus downloading without a NEON account. I am not sure if this statistic is valid all the time. Furthermore, I have written some Python code that runs on JASMIN that can read & manipulate/plot/etc. the Bundled Eddy Covariance temperature data (in HDF format) from the NEON repository. There are also the equivalents of sw\_down, lw\_down, presBaro, and wind data in this bundled file. There is also the precip data and the VERTLEVEL=000 relative humidity data that is available in separate CSV files. So I am continuing to revise the code so that it can read in all the data. This is what I have so far, for Python lists of strings, for iterating on in Python. Maybe some of these variables need adjustment? For example, should sw\_down and lw\_down be at the vertlevel=000 or vertlevel=tower top? Another example, is should pstar be at the soil surface or at vertlevel=35? The star generally means canopy top in JULES, I think. Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [13 October 2022 09:32 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/16 "2022-10-13T09:32:28Z")
```auto #tTop is an extra one for temperature at the top of the tower, t is at VERTLEVEL=000, to match the RH measurement location. #We need to compute q = specific humidity, which is not relative humidity as in the NEON files, so I think it is important to use 'tempRHMean' for this calculation of q #JULES variable names var=['t', , 'tTop', 'sw_down', 'q', 'lw_down', 'pstar', 'precip', 'wind'] #data file source va0=['RH', , 'BundleEddy', 'BundleEddy', 'RH', 'BundleEddy', 'BundleEddy', 'PRIPRE', 'BundleEddy'] #variable names in data file va1=['tempRHMean', 'tempAirTop', 'radiNet', 'RHMean', 'radiNet', 'presBaro', 'priPrecipBulk', 'soni'] #vertlevel and time for variable name va2=[None, '000_040_30m','000_040_30m',None, '000_040_30m','000_035_30m',None, '000_040_30m'] #sub-variable name va3=[None, 'temp', 'radiSwIn', None, 'radiLwIn', 'presAtm', None, 'veloXaxsYaxsErth'] #statistic of sub-variable (i.e., mean, max, min) va4=[None, 'mean', 'mean', None, 'mean', 'mean', None, 'mean'] ``` ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [13 October 2022 09:50 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/17 "2022-10-13T09:50:49Z")
Tristan suggests using all driving-data variables above the canopy. But we’re not sure right now how to define z1\_tq. The default for z1\_tq is 10 meters. But if we’re using driving-data variables all above the canopy, then maybe some sort of variable height for z1\_tq for the different sites is needed? Patrick ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [13 October 2022 11:29 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/18 "2022-10-13T11:29:03Z")
Hi Tristan: Further down in the Google Scholar search for “zero-plane jules” was this relevant article. Catherine Van den Hoof, Pier Luigi Vidale, Anne Verhoef, and Caroline Vincke “Improved evaporative flux partitioning and carbon flux in the land surface model JULES: Impact on the simulation of land surface processes in temperate Europe” ## [Agricultural and Forest Meteorology] Vol. 181 (2013) Pages 108-124 > **[Improved evaporative flux partitioning and carbon flux in the land surface...](https://www.sciencedirect.com/science/article/pii/S0168192313001913)** > > The primary role of land surface models embedded in climate models is to partition surface available energy into upwards, radiative, sensible and late… where they say: > “The distance between canopy height and the zero-plane displacement height H- > d, with d the displacement height, is a required parameter in JULES. > The values for d were taken from site specific information for Brass- > chaat and Vielsalm: 19 m (Carrara et al., 2003) and 28.5 m (Ligne > et al., 2010), respectively, or set to 2/3 h (Brutsaert, 1982) for the > other sites.” [h is the vegetation height.] ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [20 October 2022 13:58 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/19 "2022-10-20T13:58:33Z")
Hi Tristan: Here are my updated variables (now at the top of the tower: VERT=060 for SCBI) that I will try to get extracted from the HDF5 files. Note that the barometric pressure is at VERT=015, and the PRECIP is at VERT=000. ```auto #BundleEddy filename=NEON_eddy-flux-3/NEON.D02.SCBI.DP4.00200.001.2017-11.basic.20220120T173946Z.RELEASE-2022/NEON.D02.SCBI.DP4.00200.001.nsae.2017-11.basic.20211220T234551Z.h5 #PRIPRE filename=NEON_precipitation/NEON.D02.SCBI.DP1.00006.001.2017-11.basic.20220120T173946Z.RELEASE-2022/NEON.D02.SCBI.DP1.00006.001.900.000.030.PRIPRE_30min.2017-11.basic.20211210T173420Z.csv #RH #HOR.VER= 0.06 = 52 meters (tower top) filename=NEON_rel-humidity/NEON.D02.SCBI.DP1.00098.001.2017-11.basic.20220120T173946Z.RELEASE-2022/NEON.D02.SCBI.DP1.00098.001.000.060.030.RH_30min.2017-11.basic.20211210T211406Z.csv #t is at HOR.VER= 0.06, to match the RH measurement location. #We need to compute q = specific humidity, which is not relative humidity as in the NEON files, so I think it is important to use 'tempRHMean' for this calculation of q #JULES variable names for SCBI site var=['t', 'sw_down', 'q', 'lw_down', 'pstar', 'precip', 'wind'] #data file source va0=['RH', 'BundleEddy', 'RH', 'BundleEddy', 'BundleEddy', 'PRIPRE', 'BundleEddy'] #variable names in data file va1=['tempRHMean', 'radiNet', 'RHMean', 'radiNet', 'presBaro', 'priPrecipBulk', 'soni'] #vertlevel and time for variable name va2=[None, '000_060_30m',None, '000_060_30m','000_015_30m',None, '000_060_30m'] #sub-variable name va3=[None, 'radiSwIn', None, 'radiLwIn', 'presAtm', None, 'veloXaxsYaxsErth'] #statistic of sub-variable (i.e., mean, max, min) va4=[None, 'mean', None, 'mean', 'mean', None, 'mean'] ``` ---
**Author:** ![pmcguire](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@pmcguire](https://cms-helpdesk.ncas.ac.uk/u/pmcguire)\ **Post date:** [20 October 2022 15:44 UTC](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734/20 "2022-10-20T15:44:41Z")
Hi Tristan: I have a new draft of the Python code that runs and iterates over the monthly HDF5 files from the NEON SCBI site, and can read in the various top of tower variables. I still need to merge in the reading of the CSV files for temperatureRH, RH, and primary precipitation. I also need to check the units and verify that the numbers are right. Below are some print statements from the code for April and May of 2017 for the Smithsonian SCBI site in Virginia. Patrick ```auto /gws/nopw/j04/landsurf_rdg/pmcguire/TristanQ/NEON.SCBI/NEON.D02.SCBI.DP4.00200.001.2017-04.basic.20220120T173946Z.RELEASE-2022 sw_down radiNet b'2017-04-01T00:00:00.000Z' b'2017-04-30T23:30:00.000Z' Range of min data (30min samples): -4.99 983.8 Range of mean data (30min samples): -4.97 992.02 Range of max data (30min samples): -4.93 1388.18 lw_down radiNet b'2017-04-01T00:00:00.000Z' b'2017-04-30T23:30:00.000Z' Range of min data (30min samples): 225.3 410.3 Range of mean data (30min samples): 225.8 414.6 Range of max data (30min samples): 226.4 431.2 pstar presBaro b'2017-04-01T00:00:00.000Z' b'2017-04-30T23:30:00.000Z' Range of min data (30min samples): 94.92945 98.74174 Range of mean data (30min samples): 94.97379 98.75372 Range of max data (30min samples): 95.02046 98.76273 wind soni b'2017-04-01T00:00:00.000Z' b'2017-04-30T23:30:00.000Z' Range of min data (30min samples): 0.000707106815127673 6.385916815387671 Range of mean data (30min samples): 0.47491681358692645 9.21354457544177 Range of max data (30min samples): 1.031953593772511 21.11054109383807 /gws/nopw/j04/landsurf_rdg/pmcguire/TristanQ/NEON.SCBI/NEON.D02.SCBI.DP4.00200.001.2017-05.basic.20220120T173946Z.RELEASE-2022 sw_down radiNet b'2017-05-01T00:00:00.000Z' b'2017-05-31T23:30:00.000Z' Range of min data (30min samples): -4.99 1042.88 Range of mean data (30min samples): -4.99 1091.44 Range of max data (30min samples): -4.99 1547.04 lw_down radiNet b'2017-05-01T00:00:00.000Z' b'2017-05-31T23:30:00.000Z' Range of min data (30min samples): 236.5 418.7 Range of mean data (30min samples): 237.3 427.7 Range of max data (30min samples): 238.8 442.4 pstar presBaro b'2017-05-01T00:00:00.000Z' b'2017-05-31T23:30:00.000Z' Range of min data (30min samples): 95.02946 98.3237 Range of mean data (30min samples): 95.03807 98.35979 Range of max data (30min samples): 95.05346 98.37971 wind soni b'2017-05-01T00:00:00.000Z' b'2017-05-31T23:30:00.000Z' Range of min data (30min samples): 0.0003535534075638365 36.681834081692266 Range of mean data (30min samples): 0.33413772032718864 36.681834081692266 Range of max data (30min samples): 1.1196487562327306 58.91044169838825 ``` [Next page](https://cms-helpdesk.ncas.ac.uk/t/jules-neon-suite/734.md?page=2)