2024-07-23T12:42:21Z INFO __main__ 304: PIPELINE VERSION Version: 1.4.1 Common instrument config version: v0.1.4 PROCESSING VERSIONS Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-07-23T12:42:21Z INFO __main__ 269: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1 waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-07-23T12:42:21Z INFO __main__ 300: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 32807 SPICE: [PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm'), PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm')] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.37', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.37', 'version': ['2', '26', '37']}, {'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-07-23T12:42:22Z INFO stixcore.io.RidLutManager 154: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20240718/20240817 2024-07-23T12:42:22Z INFO stixcore.io.RidLutManager 163: found 1 entries 2024-07-23T12:42:23Z INFO stixcore.io.RidLutManager 172: write total 32807 entries to local storage 2024-07-23T12:42:23Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.184.14.19.10.200.DeUV@2024.205.10.00.02.217.9.xml') 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405281702-51598.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405319623-52284.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405276982-51596.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405319700-52283.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405317334-52282.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405280975-51599.fits 2024-07-23T12:42:25Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405271088-51597.fits 2024-07-23T12:42:25Z INFO __main__ 223: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405271088-51597.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405276982-51596.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405280975-51599.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405281702-51598.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405317334-52282.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405319623-52284.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2405319700-52283.fits')} 2024-07-23T12:42:25Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-07-23T12:42:25Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-07-23T12:42:26Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:26Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:26Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:26Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:26Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-07-23T12:42:26Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-07-23T12:42:27Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 10663 1 774925375 10664 0 774925437 10665 0 774925511 10666 0 774925584 10667 0 774925661 10668 0 774925730 10669 0 774925793 10670 0 774925854 10671 0 774925926 10672 0 774925982 10673 0 774926029 2024-07-23T12:42:56Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 9679 0 774864792 9680 0 774864792 9681 0 774864793 9682 0 774864794 9683 0 774864794 9684 0 774864801 9685 0 774864801 9686 0 774864802 9687 0 774864802 9688 0 774864804 9689 2 774864804 2024-07-23T12:42:57Z INFO __main__ 227: generated L0 files: [PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/24/solo_L0_stix-sci-xray-spec_0770459474-0770481370_V02_2405319700-52283.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770186613-0770188248_V02_2405280975-51599.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770109246-0770113334_V02_2405271088-51597.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770179115-0770179280_V02_2405281702-51598.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/24/solo_L0_stix-sci-xray-spec_0770437879-0770459774_V02_2405317334-52282.fits')] 2024-07-23T12:42:57Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:57Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:57Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:57Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:58Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/24/solo_L0_stix-sci-xray-spec_0770459474-0770481370_V02_2405319700-52283.fits 2024-07-23T12:42:58Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-07-23T12:42:58Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770186613-0770188248_V02_2405280975-51599.fits 2024-07-23T12:42:58Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-07-23T12:42:58Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/24/solo_L0_stix-sci-xray-spec_0770437879-0770459774_V02_2405317334-52282.fits 2024-07-23T12:42:58Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-07-23T12:42:58Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770109246-0770113334_V02_2405271088-51597.fits 2024-07-23T12:42:58Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-07-23T12:42:58Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0770179115-0770179280_V02_2405281702-51598.fits 2024-07-23T12:42:58Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-07-23T12:42:59Z INFO __main__ 231: generated L1 files: [PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/05/31/SCI/solo_L1_stix-sci-xray-spec_20240531T083508-20240531T144004_V02_2405319700-52283.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/05/28/SCI/solo_L1_stix-sci-xray-cpd_20240528T024229-20240528T024514_V02_2405281702-51598.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/05/28/SCI/solo_L1_stix-sci-xray-cpd_20240528T044727-20240528T051441_V02_2405280975-51599.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/05/27/SCI/solo_L1_stix-sci-xray-cpd_20240527T071800-20240527T082608_V02_2405271088-51597.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/05/31/SCI/solo_L1_stix-sci-xray-spec_20240531T023513-20240531T084008_V02_2405317334-52282.fits')] 2024-07-23T12:42:59Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:59Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240723_001.tm 2024-07-23T12:42:59Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:59Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240723_001.tm 2024-07-23T12:42:59Z WARNING stixcore.processing.L1toL2 99: No L2 product match for product