Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds

Updated: 12-15-2021 RULES OF THE ROAD: You are welcome to use the data 'as is', however, please inform me via email if you plan to use the dataset. There are a number of small issues with the dataset that are best discussed. We are interested in publications that use the data and derived v...

Full description

Bibliographic Details
Main Authors: Kaeppler, Stephen, Zhan, Weijia
Format: Dataset
Language:English
Published: Zenodo 2021
Subjects:
IPY
Online Access:https://dx.doi.org/10.5281/zenodo.5781251
https://zenodo.org/record/5781251
id ftdatacite:10.5281/zenodo.5781251
record_format openpolar
institution Open Polar
collection DataCite Metadata Store (German National Library of Science and Technology)
op_collection_id ftdatacite
language English
description Updated: 12-15-2021 RULES OF THE ROAD: You are welcome to use the data 'as is', however, please inform me via email if you plan to use the dataset. There are a number of small issues with the dataset that are best discussed. We are interested in publications that use the data and derived values that are presented within the dataset. If you plan to publish these results, please circulate a draft by me (SRK) and we would appreciate an offer of co-authorship or at minimum an acknowledgement. You should include the NSF funding numbers NSF AGS - 1853408 As a general warning, the data from PFISR are quite noisy and you may need to perform significant averaging to produce usable results. Again, please contact me and we can discuss this in more detail. Version v0.6.4.2021.07.12 - This was the final processed version at the time that the final report was submitted to the NSF. --------------- Previous from before ------------------ This file contains Poker Flat Incoherent Scatter Radar (PFISR) E-region Neutral Winds Data. These data correspond to monthly data files that include the E-region neutral winds and other parameters for the from March 2013-June 2019. Publications of the Joule Heating Results: https://doi.org/10.1029/2021JA029371 https://doi.org/10.1029/2021JA029719 Publication of Neutral Wind Results: Hopefully we will have something in 2021. RAW ISR Data: These data were processed from the following files found in: https://data.amisr.com/database/tmp/Kaeppler/winds/ and https://data.amisr.com/database/tmp/Kaeppler/missing_IPY.tar.gz Please note that the error on the line of sight velocities may have been overestimated in these data and we scaled them by a eVLOS/sqrt(10). Interested persons should contact Ashton Reimer or Roger Varney at SRI International for more information about these data, please see amisr.com Truthfully, the ISR data should eventually be reprocessed and then the winds algorithm run over it again. This is a step for future work. Processing Code is available upon request via email. File Documentation: Please see the change log: Purpose: This is the overarching program and functions which process the E region neutral winds from the fitted AC and LP data from PFISR. This is a conversion fo process_eregwinds_srk.py which was originally written by Nicolls into a more formal python class structure. 2017-10-05 - v0.2 The ProcessEregionNeutralWinds.py file has been validated against process_eregwinds_srk.py using 20161121.001_ac_3min-fitcal.h5, 20170301.013_ac_3min-fitcal.h5, 20170302.001_ac_3min-fitcal.h5. The program to run these is ComparePrograms.py. At this point these program match. I am going to start diverging the code base, first subtly in the Joule Heating since I found that Mike just looped over Nbeams, which isn't quite right, you need to loop over the beams that were selected. Changes from this point forward will produce different results. 2017-10-10 - v0.3.2017.10.10 Version v0.3, I made some IO changes but I may start processing some data with this version. Version v0.4 - lots of small edits made to the IO and the plotting software. It all seems to work I have also included the SNR and Ne into the monthly plots and other information. Made processing smoother. 03 13 2018 - added solar local time converion v0.4.1 - 09 08 2018 added some ability to extract out the raw electron and SNR densities for each altitude bin v0.4.2 - 10 15 2018 added in obtaining the F-region flows - want to check against the electric field. v0.4.3 - 10 29 2018 added in some more altitude into the Joule Heating so I can make better figures v0.4.4 - 11 20 2018 made some pretty major changes to IO to include consistent calculation of Pedersen conductivity from FastConductivity.py. Made some changes to the Joule heating calculation and checked formulas. It is worth checking again. v0.4.5 - 11 20 2018: added in Hall and Pedersen conductivities from fitted electron density data. v0.4.6 - 12 03 2018: Tried to fix some of the double counting and time problems in testMakeMonthlyh5 05 22 2019: added some statements to bypass the geophysical parameters. Also need in config file now. Additionally wrote in IOEregionwinds a try except statement 07 29 2019: Running the code for the 06 data reprocessed by Ashton v0.5.0 - 10-15-2019: put in some filtering on the LOS velocity discharging bad Chi square and bad error codes on the fit. v0.5.1 - 10-23-2019: changed chi square to 0.01 for lower boundary v0.5.3 - 12-02-2019: Added in that now passing in the Chi2 and Fitcode filtering by Config file Bigger change that I am scaling the AC dVlos by some sort of factor while Ashton figures this out. We decided that a conversative scaling would be to reduce the dVLOS by 1/sqrt(10). The chi square produced in the data Ashton sent me typically was around 0.01, so the uncertaintiies on the LOS velocities may be over estimated. So we are just changing this as a temporary fix while Ashton fixes the uncertainty estimation. v0.5.5 02 01 2020 - Added in calculation of Coriolis, Centrifugal, and Lorentz forcing v0.5.5 02 10 2020 - Added a correction to qvert so that way I can calculate the lorentz term. Found an error where qvert = 0 in the if statement goes to false. v0.5.5 02 15 2020 - Put in nuInscaler into the main program, scaling ALL kappas by the scaler number v0.5.6 02 28 2020 -- Added some more vlos diagostics and the calculation of the scale height. Added Altitude offset v0.5.6.2020.03.12_nuin_fracoff - testing putting in the Brekke formula for ion neutral collision frequency and took out frac v0.5.7.2020.04.10 - Put in Ashton's revised ion neutral collision frequency formulas into IO. Also wrote a testscript and at least for the file I used was only different by 2.5%. Revised where the mag data is being pulled from since the URL is deprecated Added in Kappa which is now being interpolate - plan to see where kappa =1 is located for the paper. commented out nuin scaler just so I am not chasing my tail test v0.5.7.2020.04.13_org commented back in original ion neutral collision frequency method possible mistake that not summing up properly. test v0.5.7.2020.04.13_newnuin_orgsum_noTr800 - new formula for nuin except took off Tr>800. I expect this should be almost the same as before since the formulas are basically the same. did the original sum using frac[0] and frac[1] want to see if I am underestimating v0.5.7.2020.04.13_newnuin_orgsum_yesTr800 - same as above except now including Tr>800. 'v0.5.7.2020.04.13_newnuin_newsum_noTr800' - using the new sum now and new col freq v0.5.7.2020.04.13_updatedorg - updated original uses original method but including the NO term v0.6.0.2020.04.15 -- Now think I have the new ion neutral collision frequency working and validated. Found a mistake in how I was calculating the ion neutral collision frequency that the fraction weight I was using only included the O+ and O2+ terms and not NO+ Turns out I was basically weighting by about 0.5, so I was effectively reducing the ion neutral collision frequency by about a factor of 0.5 or less... From this point forward need to start using any results from > v0.6 This revision has changed previous results signi v0.6.0.2020.04.21 -- updated to now include the temperature correction for the O2+ v0.6.1.2020.04.23 -- made a number of changes to the geomagnetic files and reprocessed from CDAweb. Wrote new code to be able to process the files from CDAweb in the new format. Also changed the geomagnetic data files v0.6.1.2020.06.07 -- changed the generation of Monthly files to hopefully be in order now Added in missingIPY files given to me by ashton, maybe improve data covarege Some work going to need to be done to make sure that all of the 10, 15, and 20 minute data are there. v0.6.1.2020.06.15_Weijia -- Updated the data for Weijia's study in particular since we are missing a lot of IPY data for 02-04 2013 and 2014. 'v0.6.2.2020.07.01' -- Updated the data with new IPY27 mode for 2013 and 2014 Ashton processed. Also now put in mechanical Joule heating term. Put in the conductance and conductivity now too. v0.6.2.2020.07.30 -- Made some changes to IO since Weijia noticed the mechanical heating terms were missing from the monthly files. v0.6.3.2020.10.19 -- Tried to elimated all extra instance of nuinscaler, and also output that variable. Added in variables To get the Ti, Tn, ion neutral collision frequency along the vertical beam for diagnostic purposes included dVest for F-region plasma drifts for Rafael v0.6.4.2020.11.20 -- Extracted some more parameters including F107 and the Hall and Pedersen Drags v0.6.4.2021.07.21 -- Final Run of data for NSF project
format Dataset
author Kaeppler, Stephen
Zhan, Weijia
spellingShingle Kaeppler, Stephen
Zhan, Weijia
Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
author_facet Kaeppler, Stephen
Zhan, Weijia
author_sort Kaeppler, Stephen
title Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
title_short Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
title_full Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
title_fullStr Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
title_full_unstemmed Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds
title_sort poker flat incoherent scatter radar (pfisr) observations of e-region neutral winds
publisher Zenodo
publishDate 2021
url https://dx.doi.org/10.5281/zenodo.5781251
https://zenodo.org/record/5781251
long_lat ENVELOPE(140.013,140.013,-66.668,-66.668)
ENVELOPE(-86.200,-86.200,-77.800,-77.800)
ENVELOPE(-62.183,-62.183,-70.717,-70.717)
geographic Pedersen
Reimer
Ashton
geographic_facet Pedersen
Reimer
Ashton
genre IPY
genre_facet IPY
op_relation https://dx.doi.org/10.5281/zenodo.2595296
op_rights Open Access
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
cc-by-4.0
info:eu-repo/semantics/openAccess
op_rightsnorm CC-BY
op_doi https://doi.org/10.5281/zenodo.5781251
https://doi.org/10.5281/zenodo.2595296
_version_ 1766046923686412288
spelling ftdatacite:10.5281/zenodo.5781251 2023-05-15T16:55:53+02:00 Poker Flat Incoherent Scatter Radar (PFISR) Observations of E-region Neutral Winds Kaeppler, Stephen Zhan, Weijia 2021 https://dx.doi.org/10.5281/zenodo.5781251 https://zenodo.org/record/5781251 en eng Zenodo https://dx.doi.org/10.5281/zenodo.2595296 Open Access Creative Commons Attribution 4.0 International https://creativecommons.org/licenses/by/4.0/legalcode cc-by-4.0 info:eu-repo/semantics/openAccess CC-BY dataset Dataset 2021 ftdatacite https://doi.org/10.5281/zenodo.5781251 https://doi.org/10.5281/zenodo.2595296 2022-02-08T17:05:41Z Updated: 12-15-2021 RULES OF THE ROAD: You are welcome to use the data 'as is', however, please inform me via email if you plan to use the dataset. There are a number of small issues with the dataset that are best discussed. We are interested in publications that use the data and derived values that are presented within the dataset. If you plan to publish these results, please circulate a draft by me (SRK) and we would appreciate an offer of co-authorship or at minimum an acknowledgement. You should include the NSF funding numbers NSF AGS - 1853408 As a general warning, the data from PFISR are quite noisy and you may need to perform significant averaging to produce usable results. Again, please contact me and we can discuss this in more detail. Version v0.6.4.2021.07.12 - This was the final processed version at the time that the final report was submitted to the NSF. --------------- Previous from before ------------------ This file contains Poker Flat Incoherent Scatter Radar (PFISR) E-region Neutral Winds Data. These data correspond to monthly data files that include the E-region neutral winds and other parameters for the from March 2013-June 2019. Publications of the Joule Heating Results: https://doi.org/10.1029/2021JA029371 https://doi.org/10.1029/2021JA029719 Publication of Neutral Wind Results: Hopefully we will have something in 2021. RAW ISR Data: These data were processed from the following files found in: https://data.amisr.com/database/tmp/Kaeppler/winds/ and https://data.amisr.com/database/tmp/Kaeppler/missing_IPY.tar.gz Please note that the error on the line of sight velocities may have been overestimated in these data and we scaled them by a eVLOS/sqrt(10). Interested persons should contact Ashton Reimer or Roger Varney at SRI International for more information about these data, please see amisr.com Truthfully, the ISR data should eventually be reprocessed and then the winds algorithm run over it again. This is a step for future work. Processing Code is available upon request via email. File Documentation: Please see the change log: Purpose: This is the overarching program and functions which process the E region neutral winds from the fitted AC and LP data from PFISR. This is a conversion fo process_eregwinds_srk.py which was originally written by Nicolls into a more formal python class structure. 2017-10-05 - v0.2 The ProcessEregionNeutralWinds.py file has been validated against process_eregwinds_srk.py using 20161121.001_ac_3min-fitcal.h5, 20170301.013_ac_3min-fitcal.h5, 20170302.001_ac_3min-fitcal.h5. The program to run these is ComparePrograms.py. At this point these program match. I am going to start diverging the code base, first subtly in the Joule Heating since I found that Mike just looped over Nbeams, which isn't quite right, you need to loop over the beams that were selected. Changes from this point forward will produce different results. 2017-10-10 - v0.3.2017.10.10 Version v0.3, I made some IO changes but I may start processing some data with this version. Version v0.4 - lots of small edits made to the IO and the plotting software. It all seems to work I have also included the SNR and Ne into the monthly plots and other information. Made processing smoother. 03 13 2018 - added solar local time converion v0.4.1 - 09 08 2018 added some ability to extract out the raw electron and SNR densities for each altitude bin v0.4.2 - 10 15 2018 added in obtaining the F-region flows - want to check against the electric field. v0.4.3 - 10 29 2018 added in some more altitude into the Joule Heating so I can make better figures v0.4.4 - 11 20 2018 made some pretty major changes to IO to include consistent calculation of Pedersen conductivity from FastConductivity.py. Made some changes to the Joule heating calculation and checked formulas. It is worth checking again. v0.4.5 - 11 20 2018: added in Hall and Pedersen conductivities from fitted electron density data. v0.4.6 - 12 03 2018: Tried to fix some of the double counting and time problems in testMakeMonthlyh5 05 22 2019: added some statements to bypass the geophysical parameters. Also need in config file now. Additionally wrote in IOEregionwinds a try except statement 07 29 2019: Running the code for the 06 data reprocessed by Ashton v0.5.0 - 10-15-2019: put in some filtering on the LOS velocity discharging bad Chi square and bad error codes on the fit. v0.5.1 - 10-23-2019: changed chi square to 0.01 for lower boundary v0.5.3 - 12-02-2019: Added in that now passing in the Chi2 and Fitcode filtering by Config file Bigger change that I am scaling the AC dVlos by some sort of factor while Ashton figures this out. We decided that a conversative scaling would be to reduce the dVLOS by 1/sqrt(10). The chi square produced in the data Ashton sent me typically was around 0.01, so the uncertaintiies on the LOS velocities may be over estimated. So we are just changing this as a temporary fix while Ashton fixes the uncertainty estimation. v0.5.5 02 01 2020 - Added in calculation of Coriolis, Centrifugal, and Lorentz forcing v0.5.5 02 10 2020 - Added a correction to qvert so that way I can calculate the lorentz term. Found an error where qvert = 0 in the if statement goes to false. v0.5.5 02 15 2020 - Put in nuInscaler into the main program, scaling ALL kappas by the scaler number v0.5.6 02 28 2020 -- Added some more vlos diagostics and the calculation of the scale height. Added Altitude offset v0.5.6.2020.03.12_nuin_fracoff - testing putting in the Brekke formula for ion neutral collision frequency and took out frac v0.5.7.2020.04.10 - Put in Ashton's revised ion neutral collision frequency formulas into IO. Also wrote a testscript and at least for the file I used was only different by 2.5%. Revised where the mag data is being pulled from since the URL is deprecated Added in Kappa which is now being interpolate - plan to see where kappa =1 is located for the paper. commented out nuin scaler just so I am not chasing my tail test v0.5.7.2020.04.13_org commented back in original ion neutral collision frequency method possible mistake that not summing up properly. test v0.5.7.2020.04.13_newnuin_orgsum_noTr800 - new formula for nuin except took off Tr>800. I expect this should be almost the same as before since the formulas are basically the same. did the original sum using frac[0] and frac[1] want to see if I am underestimating v0.5.7.2020.04.13_newnuin_orgsum_yesTr800 - same as above except now including Tr>800. 'v0.5.7.2020.04.13_newnuin_newsum_noTr800' - using the new sum now and new col freq v0.5.7.2020.04.13_updatedorg - updated original uses original method but including the NO term v0.6.0.2020.04.15 -- Now think I have the new ion neutral collision frequency working and validated. Found a mistake in how I was calculating the ion neutral collision frequency that the fraction weight I was using only included the O+ and O2+ terms and not NO+ Turns out I was basically weighting by about 0.5, so I was effectively reducing the ion neutral collision frequency by about a factor of 0.5 or less... From this point forward need to start using any results from > v0.6 This revision has changed previous results signi v0.6.0.2020.04.21 -- updated to now include the temperature correction for the O2+ v0.6.1.2020.04.23 -- made a number of changes to the geomagnetic files and reprocessed from CDAweb. Wrote new code to be able to process the files from CDAweb in the new format. Also changed the geomagnetic data files v0.6.1.2020.06.07 -- changed the generation of Monthly files to hopefully be in order now Added in missingIPY files given to me by ashton, maybe improve data covarege Some work going to need to be done to make sure that all of the 10, 15, and 20 minute data are there. v0.6.1.2020.06.15_Weijia -- Updated the data for Weijia's study in particular since we are missing a lot of IPY data for 02-04 2013 and 2014. 'v0.6.2.2020.07.01' -- Updated the data with new IPY27 mode for 2013 and 2014 Ashton processed. Also now put in mechanical Joule heating term. Put in the conductance and conductivity now too. v0.6.2.2020.07.30 -- Made some changes to IO since Weijia noticed the mechanical heating terms were missing from the monthly files. v0.6.3.2020.10.19 -- Tried to elimated all extra instance of nuinscaler, and also output that variable. Added in variables To get the Ti, Tn, ion neutral collision frequency along the vertical beam for diagnostic purposes included dVest for F-region plasma drifts for Rafael v0.6.4.2020.11.20 -- Extracted some more parameters including F107 and the Hall and Pedersen Drags v0.6.4.2021.07.21 -- Final Run of data for NSF project Dataset IPY DataCite Metadata Store (German National Library of Science and Technology) Pedersen ENVELOPE(140.013,140.013,-66.668,-66.668) Reimer ENVELOPE(-86.200,-86.200,-77.800,-77.800) Ashton ENVELOPE(-62.183,-62.183,-70.717,-70.717)