2024-09-24T19:15:14Z INFO __main__ 304: PIPELINE VERSION Version: 1.4.1 Common instrument config version: v0.1.4 PROCESSING VERSIONS Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-09-24T19:15:14Z INFO __main__ 269: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1 waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-09-24T19:15:14Z INFO __main__ 300: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 33320 SPICE: [PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240923_001.tm'), PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240923_001.tm')] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.37', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.37', 'version': ['2', '26', '37']}, {'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-09-24T19:15:15Z INFO stixcore.io.RidLutManager 154: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20240905/20241005 2024-09-24T19:15:15Z INFO stixcore.io.RidLutManager 163: found 2 entries 2024-09-24T19:15:16Z INFO stixcore.io.RidLutManager 172: write total 33320 entries to local storage 2024-09-24T19:15:16Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.234.08.00.27.739.pMnC@2024.268.16.00.00.943.19.xml') 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407282826-58755.fits exists appending data 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407282826-58755.fits 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407287541-58760.fits exists appending data 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407287541-58760.fits 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407280402-58757.fits exists appending data 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407280402-58757.fits 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407289575-58761.fits exists appending data 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407241679-58983.fits exists appending data 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407289575-58761.fits 2024-09-24T19:15:20Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407241679-58983.fits 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407221990-58981.fits exists appending data 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407288157-58756.fits exists appending data 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407221990-58981.fits 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407256782-58984.fits exists appending data 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407256782-58984.fits 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407240790-58982.fits exists appending data 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407240790-58982.fits 2024-09-24T19:15:21Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407288157-58756.fits 2024-09-24T19:15:22Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407281995-58758.fits exists appending data 2024-09-24T19:15:22Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407281995-58758.fits 2024-09-24T19:15:22Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407285607-58759.fits exists appending data 2024-09-24T19:15:22Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407285607-58759.fits 2024-09-24T19:15:22Z INFO __main__ 223: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407280402-58757.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407281995-58758.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407282826-58755.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407285607-58759.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407287541-58760.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407288157-58756.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407289575-58761.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407221990-58981.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407240790-58982.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407241679-58983.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/24/solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407256782-58984.fits')} 2024-09-24T19:15:22Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-09-24T19:15:22Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-09-24T19:15:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240923_001.tm 2024-09-24T19:15:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240923_001.tm 2024-09-24T19:15:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240923_001.tm 2024-09-24T19:15:22Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-09-24T19:15:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240923_001.tm 2024-09-24T19:15:22Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-09-24T19:15:24Z INFO stixcore.io.fits.processors 534: Fits file solo_L0_stix-sci-xray-cpd_0775487774-0775488708_V02_2407282826-58755.fits exists appending data 2024-09-24T19:15:24Z INFO stixcore.processing.LBtoL0 121: Tried to combine 2 BSD products: sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 15127 1 780399008 15128 0 780399051 15129 0 780399096 15130 0 780399142 15131 0 780399191 15132 0 780399250 15133 0 780399316 15134 0 780399380 15135 0 780399796 15136 0 780399888 15137 0 780399944 15138 0 780399999 15139 0 780400043 15140 0 780400088 15141 0 780400126 15142 0 780400187 15143 0 780400233 15144 0 780400270 15145 0 780400312 15146 0 780400351 15147 0 780400383 15148 0 780400423 15149 0 780400466 15150 0 780400519 15151 0 780400574 15152 0 780400617 15153 0 780400661 15154 0 780400735 15155 0 780400883 15156 0 780401035 15157 0 780401158 15158 0 780401267 15159 0 780401381 15160 0 780401488 15161 0 780401603 15162 0 780401771 15163 0 780402145 15164 0 780402326 15165 0 780402459 15166 0 780402640 15167 0 780402766 15168 0 780402874 15169 0 780402984 15170 0 780403092 15171 0 780403260 15172 0 780403355 15173 0 780403442 15174 0 780403591 15175 0 780404020 15176 0 780404215 15177 0 780404333 15178 0 780404447 15179 0 780404570 15180 0 780404663 15181 0 780404743 15182 0 780404822 15183 0 780404906 15184 0 780404982 15185 0 780405061 15186 0 780405132 15187 0 780405206 15188 0 780405283 15189 0 780405352 15190 0 780405415 15191 0 780405486 15192 0 780405550 15193 0 780405619 15194 0 780405695 15195 0 780405754 15196 0 780405812 15197 0 780405868 15198 0 780405926 15199 0 780405981 15200 0 780406030 15201 0 780406086 15202 0 780406140 15203 0 780406192 15204 0 780406258 15205 0 780406322 15206 0 780406392 15207 0 780406479 15208 0 780406710 15209 0 780407144 15210 0 780407771 15211 0 780407860 15212 0 780407923 15213 0 780407977 15214 0 780408028 15215 0 780408066 15216 0 780408109 15217 0 780408151 15218 0 780408186 2024-09-24T19:15:32Z INFO stixcore.io.fits.processors 534: Fits file solo_L0_stix-sci-xray-spec_0775149956-0775171856_V02_2407240790-58982.fits exists appending data 2024-09-24T19:15:32Z INFO stixcore.processing.LBtoL0 121: Tried to combine 2 BSD products: