2025-03-25T18:18:02Z INFO __main__ 300:
PIPELINE VERSION
Version: 1.5.0
Common instrument config version: v0.1.5
PROCESSING VERSIONS
Prod: Ephemeris
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/ANC/aspect.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py
Vers: 2
Prod: RawPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: CompressedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: SummedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Visibility
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Spectrogram
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: Aspect
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py
Vers: 2
Prod: RawPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: CompressedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: SummedPixelData
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Visibility
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Spectrogram
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: Aspect
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py
Vers: 2
Prod: MiniReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py
Vers: 2
Prod: MaxiReport
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Background
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Spectra
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: Variance
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: EnergyCalibration
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: TMStatusFlareList
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py
Vers: 2
Prod: LevelB
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py
Vers: 2
Prod: LightCurve
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py
Vers: 2
Prod: FlareFlag
File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py
Vers: 2
2025-03-25T18:18:02Z INFO __main__ 265:
CONFIG
[Paths]
tm_archive = /data/stix/SOLSOC/from_edds/tm/processing
fits_archive = /data/stix/out/fits_v1.2.0
spice_kernels = /data/stix/spice/kernels
soop_files = /data/stix/SOLSOC/from_soc
[Logging]
stop_on_error = False
[IDLBridge]
enabled = True
gsw_path = /opt/STIX-GSW
batchsize = 30
[Pipeline]
status_server_port = 12388
parallel_batchsize_l0 = 100
parallel_batchsize_l1 = 100
parallel_batchsize_l2 = 100
log_level = INFO
log_dir = /data/stix/out/fits_v1.2.0/logging
error_mail_send = True
error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie
error_mail_smpt_host = lmailer.fhnw.ch
error_mail_smpt_port = 25
error_mail_sender = nicky.hochmuth@fhnw.ch
sync_tm_at_start = True
stop_after_unprocessed = True
start_with_unprocessed = True
last_processed = TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.184.14.18.45.363.hBFL@2024.200.08.00.00.905.1.xml.out
history_db_file = /home/stixcore/soar_pub/processing_history.sqlite
[Publish]
db_file = /home/stixcore/soar_pub/published.sqlite
include_levels = L0, L1, ANC
waiting_period = 21d
target_dir = /home/stixcore/from_pub099
report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie
report_mail_send = True
same_esa_name_dir = /data/stix/out/esa_conflicts
rid_lut_file = ./stixcore/data/publish/rid_lut.csv
rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/
batch_size = 20000
fits_dir = /data/stix/out/fits_v1.2.0
target_host = pub026.cs.technik.fhnw.ch
supplement_report = /home/stixcore/soar_pub/supplement_report.csv
include_products = ql,hk,sci,cal,asp
log_dir = /data/stix/out/fits_v1.2.0/logging
[SOOP]
endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api
user = smaloney
password = xxxx
soop_files_download = ./stixcore/data/soop
2025-03-25T18:18:02Z INFO __main__ 296:
SINGLETONS
SOOPManager: /data/stix/SOLSOC/from_soc
RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 34987
SPICE: [(PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250324_003.tm'), 'pred', datetime.datetime(2025, 3, 24, 0, 0)), (PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250325_001.tm'), 'flown', datetime.datetime(2025, 3, 25, 0, 0))]
IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb
Versions:
[{'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}]
Force version: None
History:
IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')])
2025-03-25T18:18:03Z INFO stixcore.io.RidLutManager 176: write total 34987 entries to local storage
2025-03-25T18:18:03Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250324_003.tm
2025-03-25T18:18:03Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250325_002.tm
2025-03-25T18:18:03Z INFO __main__ 209: new spice kernels detected and loaded
2025-03-25T18:18:03Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_Hourly_Req_BatchRequest.PktTmRaw.SOL.0.2025.033.09.50.48.622.yWGN@2025.084.13.00.02.709.2.xml')
2025-03-25T18:18:09Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502245116-63110.fits
2025-03-25T18:18:09Z INFO stixcore.io.fits.processors 448: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502241572-63111.fits exists appending data
2025-03-25T18:18:09Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502241572-63111.fits
2025-03-25T18:18:09Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502248878-63107.fits
2025-03-25T18:18:09Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502242131-63108.fits
2025-03-25T18:18:10Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502244245-63109.fits
2025-03-25T18:18:10Z INFO __main__ 218: generated LB files:
{PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502241572-63111.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502242131-63108.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502244245-63109.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502245116-63110.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2502248878-63107.fits')}
2025-03-25T18:18:10Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite
2025-03-25T18:18:10Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250324_003.tm
2025-03-25T18:18:10Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250325_002.tm
2025-03-25T18:18:10Z INFO stixcore.io.RidLutManager 178: read rid-lut from stixcore/data/publish/rid_lut.csv
2025-03-25T18:18:11Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence <LevelB LB, 21, 6, 21 796177507:8623-796177557:48148>
sequence_count sequence_flag scet_coarse
-------------- ------------- -----------
7479 0 796177507
7480 0 796177507
7481 0 796177508
7482 0 796177508
7483 0 796177509
7484 0 796177509
7485 0 796177510
7486 0 796177511
7487 0 796177511
7488 0 796177512
7489 0 796177512
7490 0 796177513
7491 0 796177513
7492 0 796177513
7493 0 796177514
7494 0 796177514
7495 0 796177515
7496 0 796177515
7497 0 796177516
7498 0 796177517
7499 0 796177517
7500 0 796177518
7501 0 796177519
7502 0 796177519
7503 0 796177519
7504 0 796177520
7505 0 796177520
7506 0 796177521
7507 0 796177521
7508 0 796177522
7509 0 796177523
7510 0 796177523
7511 0 796177524
7512 0 796177524
7513 0 796177525
7514 0 796177525
7515 0 796177526
7516 0 796177527
7517 0 796177527
7518 0 796177528
7519 0 796177528
7520 0 796177529
7521 0 796177529
7522 0 796177530
7523 0 796177531
7524 0 796177531
7525 0 796177532
7526 0 796177532
7527 0 796177533
7528 0 796177533
7529 0 796177534
7530 0 796177535
7531 0 796177535
7532 0 796177536
7533 0 796177536
7534 0 796177537
7535 0 796177537
7536 0 796177538
7537 0 796177538
7538 0 796177539
7539 0 796177539
7540 0 796177540
7541 0 796177541
7542 0 796177541
7543 0 796177542
7544 0 796177543
7545 0 796177543
7546 0 796177544
7547 0 796177544
7548 0 796177544
7549 0 796177545
7550 0 796177546
7551 0 796177546
7552 0 796177547
7553 0 796177547
7554 0 796177548
7555 0 796177548
7556 0 796177549
7557 0 796177549
7558 0 796177550
7559 0 796177551
7560 0 796177551
7561 0 796177552
7562 0 796177552
7563 0 796177553
7564 0 796177553
7565 0 796177554
7566 0 796177555
7567 0 796177556
7568 0 796177556
7569 0 796177557
7570 0 796177557
7571 2 796177557
2025-03-25T18:21:26Z INFO __main__ 222: generated L0 files:
[PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793716021-0793718437_V02_2502244245-63109.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793711046-0793714663_V02_2502242131-63108.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793725589-0793727742_V02_2502241572-63111.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793715921-0793725861_V02_2502245116-63110.fits')]
2025-03-25T18:21:26Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20250324_003.tm
2025-03-25T18:21:26Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20250325_002.tm
2025-03-25T18:21:26Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793716021-0793718437_V02_2502244245-63109.fits
2025-03-25T18:21:27Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 11A, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-02-23 18:30:00+00:00, 2025-03-08 00:00:00+00:00>
2025-03-25T18:21:27Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793711046-0793714663_V02_2502242131-63108.fits
2025-03-25T18:21:27Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 11A, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-02-23 18:30:00+00:00, 2025-03-08 00:00:00+00:00>
2025-03-25T18:21:27Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793725589-0793727742_V02_2502241572-63111.fits
2025-03-25T18:21:28Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 11A, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-02-23 18:30:00+00:00, 2025-03-08 00:00:00+00:00>
2025-03-25T18:21:28Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0793715921-0793725861_V02_2502245116-63110.fits
2025-03-25T18:21:28Z INFO stixcore.soop.manager 484: Soop found: SOOP<RF6, 11A, R_FULL_LRES_LCAD_RS-Synoptics-High, 2025-02-23 18:30:00+00:00, 2025-03-08 00:00:00+00:00>
2025-03-25T18:21:28Z INFO __main__ 226: generated L1 files:
[PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/02/24/SCI/solo_L1_stix-sci-xray-cpd_20250224T124317-20250224T152858_V02_2502245116-63110.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/02/24/SCI/solo_L1_stix-sci-xray-cpd_20250224T124457-20250224T132514_V02_2502244245-63109.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/02/24/SCI/solo_L1_stix-sci-xray-cpd_20250224T152425-20250224T160019_V02_2502241572-63111.fits'),
PosixPath('/data/stix/out/fits_v1.2.0/L1/2025/02/24/SCI/solo_L1_stix-sci-xray-cpd_20250224T112203-20250224T122220_V02_2502242131-63108.fits')]