2024-07-06T09:47:35Z INFO __main__ 304: PIPELINE VERSION Version: 1.4.1 Common instrument config version: v0.1.4 PROCESSING VERSIONS Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-07-06T09:47:35Z INFO __main__ 269: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1 waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-07-06T09:47:35Z INFO __main__ 300: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 32786 SPICE: [PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240620_006.tm'), PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240705_002.tm')] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.37', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.37', 'version': ['2', '26', '37']}, {'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-07-06T09:47:36Z INFO stixcore.io.RidLutManager 154: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20240629/20240729 2024-07-06T09:47:36Z INFO stixcore.io.RidLutManager 163: found 1 entries 2024-07-06T09:47:37Z INFO stixcore.io.RidLutManager 172: write total 32786 entries to local storage 2024-07-06T09:47:37Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.184.14.18.06.671.JJMh@2024.188.07.00.03.863.8.xml') 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125007-50136.fits exists appending data 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125007-50136.fits 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128491-50142.fits exists appending data 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128491-50142.fits 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405127257-50138.fits exists appending data 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405127257-50138.fits 2024-07-06T09:47:39Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125155-50135.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125155-50135.fits 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405080917-50131.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405080917-50131.fits 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123352-50140.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123352-50140.fits 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128163-50141.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128163-50141.fits 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405082297-50132.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405082297-50132.fits 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123626-50134.fits exists appending data 2024-07-06T09:47:40Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123626-50134.fits 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129320-50139.fits exists appending data 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129320-50139.fits 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129798-50137.fits exists appending data 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129798-50137.fits 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405136481-50143.fits exists appending data 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405136481-50143.fits 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405081981-50133.fits exists appending data 2024-07-06T09:47:41Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405081981-50133.fits 2024-07-06T09:47:41Z INFO __main__ 223: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405080917-50131.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405081981-50133.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405082297-50132.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123352-50140.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405123626-50134.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125007-50136.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405125155-50135.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405127257-50138.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128163-50141.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405128491-50142.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129320-50139.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405129798-50137.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405136481-50143.fits')} 2024-07-06T09:47:41Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-07-06T09:47:41Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240620_006.tm 2024-07-06T09:47:42Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240705_002.tm 2024-07-06T09:47:42Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-07-06T09:47:42Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 6949 1 772965095 6950 0 772965111 6951 0 772965137 6952 0 772965169 6953 0 772965181 6954 0 772965193 6955 0 772965196 6956 0 772965206 6957 0 772965206 6958 0 772965210 6959 0 772965214 6960 0 772965226 6961 0 772965243 6962 0 772965268 6963 0 772965281 6964 0 772965287 6965 0 772965299 6966 0 772965299 6967 0 772965303 6968 0 772965315 6969 0 772965332 6970 0 772965348 6971 0 772965361 6972 0 772965400 6973 0 772965434 6974 0 772965435 6975 0 772965464 6976 0 772965491 6977 0 772965507 6978 0 772965517 6979 0 772965533 6980 0 772965551 6981 0 772965581 6982 0 772965626 6983 0 772965626 6984 0 772965670 6985 0 772965715 6986 0 772965756 6987 0 772965801 6988 0 772965834 6989 0 772965860 6990 0 772965877 6991 0 772965877 6992 0 772965911 6993 0 772965941 6994 0 772965966 6995 0 772965984 6996 0 772966000 6997 0 772966013 6998 0 772966027 6999 0 772966042 7000 0 772966043 7001 0 772966055 7002 0 772966071 7003 0 772966087 7004 0 772966105 7005 0 772966136 7006 0 772966165 7007 0 772966196 7008 0 772966196 7009 0 772966218 7010 0 772966224 7011 0 772966236 7012 0 772966240 7013 0 772966251 7014 0 772966255 7015 0 772966265 7016 0 772966270 7017 0 772966270 7018 0 772966284 7019 0 772966299 7020 0 772966311 7021 0 772966327 7022 0 772966333 7023 0 772966344 7024 0 772966348 7025 0 772966348 7026 0 772966364 7027 0 772966406 7028 0 772966436 7029 0 772966454 7030 0 772966469 7031 0 772966483 7032 0 772966495 7033 0 772966499 7034 0 772966500 7035 0 772966511 7036 0 772966525 7037 0 772966529 7038 0 772966540 7039 0 772966544 7040 0 772966556 7041 0 772966571 7042 0 772966571 7043 0 772966576 7044 0 772966587 7045 0 772966598 7046 0 772966602 7047 0 772966613 7048 0 772966617 7049 0 772966628 7050 0 772966632 7051 0 772966632 7052 0 772966644 7053 0 772966648 7054 0 772966663 7055 0 772966674 7056 0 772966681 7057 0 772966692 7058 0 772966696 7059 0 772966696 7060 0 772966706 7061 0 772966718 7062 0 772966723 7063 0 772966736 7064 0 772966749 7065 0 772966755 7066 0 772966765 2024-07-06T09:47:43Z INFO stixcore.io.fits.processors 534: Fits file solo_L0_stix-sci-xray-cpd_0768872713-0768873413_V02_2405128491-50142.fits exists appending data 2024-07-06T09:47:43Z INFO stixcore.processing.LBtoL0 121: Tried to combine 2 BSD products: