2025-02-20T12:00:21Z INFO __main__ 300:
PIPELINE VERSION
Version: 1.5.0
Common instrument config version: v0.1.5
PROCESSING VERSIONS
Prod: Ephemeris
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/ANC/aspect.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: RawPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: CompressedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: SummedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Visibility
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Spectrogram
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Aspect
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: RawPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: CompressedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: SummedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Visibility
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Spectrogram
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Aspect
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: LevelB
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py
Vers: 2
2025-02-20T12:00:21Z INFO __main__ 265:
CONFIG
[Paths]
tm_archive = /data/stix/SOLSOC/from_edds/tm/processing
fits_archive = /data/stix/out/fits_v1.2.0
spice_kernels = /data/stix/spice/kernels
soop_files = /data/stix/SOLSOC/from_soc
[Logging]
stop_on_error = False
[IDLBridge]
enabled = True
gsw_path = /opt/STIX-GSW
batchsize = 30
[Pipeline]
status_server_port = 12388
parallel_batchsize_l0 = 100
parallel_batchsize_l1 = 100
parallel_batchsize_l2 = 100
log_level = INFO
log_dir = /data/stix/out/fits_v1.2.0/logging
error_mail_send = True
error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie
error_mail_smpt_host = lmailer.fhnw.ch
error_mail_smpt_port = 25
error_mail_sender = nicky.hochmuth@fhnw.ch
sync_tm_at_start = True
stop_after_unprocessed = True
start_with_unprocessed = True
last_processed = TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.184.14.18.45.363.hBFL@2024.200.08.00.00.905.1.xml.out
history_db_file = /home/stixcore/soar_pub/processing_history.sqlite
[Publish]
db_file = /home/stixcore/soar_pub/published.sqlite
include_levels = L0, L1, ANC
waiting_period = 21d
target_dir = /home/stixcore/from_pub099
report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie
report_mail_send = True
same_esa_name_dir = /data/stix/out/esa_conflicts
rid_lut_file = ./stixcore/data/publish/rid_lut.csv
rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/
batch_size = 20000
fits_dir = /data/stix/out/fits_v1.2.0
target_host = pub026.cs.technik.fhnw.ch
supplement_report = /home/stixcore/soar_pub/supplement_report.csv
include_products = ql,hk,sci,cal,asp
log_dir = /data/stix/out/fits_v1.2.0/logging
[SOOP]
endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api
user = smaloney
password = xxxx
soop_files_download = ./stixcore/data/soop
2025-02-20T12:00:21Z INFO __main__ 296:
SINGLETONS
SOOPManager: /data/stix/SOLSOC/from_soc
RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 34730
SPICE: [(PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250211_002.tm'), 'pred', datetime.datetime(2025, 2, 11, 0, 0)), (PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250220_002.tm'), 'flown', datetime.datetime(2025, 2, 20, 0, 0))]
IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb
Versions:
[{'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}]
Force version: None
History:
IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')])
2025-02-20T12:00:22Z INFO stixcore.io.RidLutManager 158: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20250215/20250317
2025-02-20T12:00:23Z INFO stixcore.io.RidLutManager 176: write total 34730 entries to local storage
2025-02-20T12:00:23Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_Hourly_Req_BatchRequest.PktTmRaw.SOL.0.2025.033.09.49.40.608.xtPo@2025.051.06.00.03.705.2.xml')
2025-02-20T12:00:30Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501315228-59115.fits
2025-02-20T12:00:31Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501274928-59114.fits
2025-02-20T12:00:32Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501254352-59112.fits
2025-02-20T12:00:32Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501252593-59113.fits
2025-02-20T12:00:32Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502011504-59116.fits
2025-02-20T12:00:32Z INFO __main__ 218: generated LB files:
{PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501252593-59113.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501254352-59112.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501274928-59114.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2501315228-59115.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502011504-59116.fits')}
2025-02-20T12:00:33Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite
2025-02-20T12:00:33Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250211_002.tm
2025-02-20T12:00:33Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250220_002.tm
2025-02-20T12:00:33Z INFO stixcore.io.RidLutManager 178: read rid-lut from stixcore/data/publish/rid_lut.csv
2025-02-20T12:00:35Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence <LevelB LB, 21, 6, 21 793334749:5283-793334943:60829>
sequence_count sequence_flag scet_coarse
-------------- ------------- -----------
7130 0 793334749
7131 0 793334759
7132 0 793334761
7133 0 793334768
7134 0 793334771
7135 0 793334779
7136 0 793334782
7137 0 793334785
7138 0 793334794
7139 0 793334796
7140 0 793334799
7141 0 793334799
7142 0 793334801
7143 0 793334810
7144 0 793334812
7145 0 793334815
7146 0 793334817
7147 0 793334826
7148 0 793334829
7149 0 793334830
7150 0 793334834
7151 0 793334843
7152 0 793334845
7153 0 793334848
7154 0 793334850
7155 0 793334859
7156 0 793334861
7157 0 793334863
7158 0 793334863
7159 0 793334865
7160 0 793334874
7161 0 793334876
7162 0 793334878
7163 0 793334880
7164 0 793334882
7165 0 793334890
7166 0 793334891
7167 0 793334893
7168 0 793334895
7169 0 793334897
7170 0 793334899
7171 0 793334907
7172 0 793334909
7173 0 793334911
7174 0 793334913
7175 0 793334914
7176 0 793334916
7177 0 793334924
7178 0 793334927
7179 0 793334928
7180 0 793334930
7181 0 793334931
7182 0 793334933
7183 0 793334933
7184 0 793334940
7185 0 793334942
7186 0 793334943
7187 2 793334943
2025-02-20T12:04:21Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence <LevelB LB, 21, 6, 21 793340495:45289-793340505:291>
sequence_count sequence_flag scet_coarse
-------------- ------------- -----------
9662 1 793340495
9663 0 793340496
9664 0 793340496
9665 0 793340497
9666 0 793340498
9667 0 793340498
9668 0 793340499
9669 0 793340499
9670 0 793340500
9671 0 793340500
9672 0 793340501
9673 0 793340502
9674 0 793340502
9675 0 793340503
9676 0 793340504
9677 0 793340504
9678 0 793340505
2025-02-20T12:04:22Z INFO __main__ 222: generated L0 files:
[PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791278510-0791283700_V02_2501274928-59114.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791104644-0791105890_V02_2501252593-59113.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791601713-0791602516_V02_2501315228-59115.fits')]
2025-02-20T12:04:22Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250211_002.tm
2025-02-20T12:04:22Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250220_002.tm
2025-02-20T12:04:23Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791278510-0791283700_V02_2501274928-59114.fits
2025-02-20T12:04:23Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 116, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-01-27 00:00:00+00:00, 2025-02-03 00:00:00+00:00>
2025-02-20T12:04:24Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791104644-0791105890_V02_2501252593-59113.fits
2025-02-20T12:04:24Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 115, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-01-20 00:00:00+00:00, 2025-01-27 00:00:00+00:00>
2025-02-20T12:04:24Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0791601713-0791602516_V02_2501315228-59115.fits
2025-02-20T12:04:24Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 116, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-01-27 00:00:00+00:00, 2025-02-03 00:00:00+00:00>
2025-02-20T12:04:24Z INFO __main__ 226: generated L1 files:
[PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/01/31/SCI/solo_L1_stix-sci-xray-cpd_20250131T012626-20250131T013949_V02_2501315228-59115.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/01/27/SCI/solo_L1_stix-sci-xray-cpd_20250127T073943-20250127T090612_V02_2501274928-59114.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/01/25/SCI/solo_L1_stix-sci-xray-cpd_20250125T072156-20250125T074242_V02_2501252593-59113.fits')]