2024-12-22T00:15:29Z INFO __main__ 300: PIPELINE VERSION Version: 1.5.0 Common instrument config version: v0.1.5 PROCESSING VERSIONS Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/ANC/aspect.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-12-22T00:15:29Z INFO __main__ 265: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch sync_tm_at_start = True stop_after_unprocessed = True start_with_unprocessed = True last_processed = TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.184.14.18.45.363.hBFL@2024.200.08.00.00.905.1.xml.out history_db_file = /home/stixcore/soar_pub/processing_history.sqlite [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1, ANC waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-12-22T00:15:29Z INFO __main__ 296: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 33782 SPICE: [(PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20241213_002.tm'), 'pred', datetime.datetime(2024, 12, 13, 0, 0)), (PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20241221_002.tm'), 'flown', datetime.datetime(2024, 12, 21, 0, 0))] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-12-22T00:15:30Z INFO stixcore.io.RidLutManager 158: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20241215/20250114 2024-12-22T00:15:30Z INFO stixcore.io.RidLutManager 167: found 6 entries 2024-12-22T00:15:31Z INFO stixcore.io.RidLutManager 176: write total 33782 entries to local storage 2024-12-22T00:15:31Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.54.04.509.INkE@2024.356.18.00.01.780.4.xml') 2024-12-22T00:15:36Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411052192-50668.fits 2024-12-22T00:15:37Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411054602-50669.fits 2024-12-22T00:15:37Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411055007-50670.fits 2024-12-22T00:15:37Z INFO stixcore.io.fits.processors 448: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411010773-50671.fits exists appending data 2024-12-22T00:15:37Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411010773-50671.fits 2024-12-22T00:15:38Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411050742-50667.fits 2024-12-22T00:15:38Z INFO __main__ 218: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411010773-50671.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411050742-50667.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411052192-50668.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411054602-50669.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2411055007-50670.fits')} 2024-12-22T00:15:38Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-12-22T00:15:38Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20241213_002.tm 2024-12-22T00:15:38Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20241221_002.tm 2024-12-22T00:15:38Z INFO stixcore.io.RidLutManager 178: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-12-22T00:19:00Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 14151 0 788095847 14152 0 788095849 14153 0 788095851 14154 0 788095860 14155 0 788095862 14156 0 788095864 14157 0 788095864 14158 0 788095866 14159 0 788095869 14160 0 788095878 14161 0 788095881 14162 0 788095883 14163 0 788095886 14164 0 788095895 14165 0 788095898 14166 0 788095898 14167 0 788095901 14168 0 788095911 14169 0 788095916 14170 0 788095926 14171 0 788095929 14172 0 788095934 14173 0 788095937 14174 0 788095937 14175 0 788095946 14176 0 788095947 14177 0 788095950 14178 0 788095952 14179 0 788095954 14180 0 788095955 14181 0 788095964 14182 0 788095966 14183 0 788095966 14184 0 788095968 14185 0 788095970 14186 0 788095971 14187 0 788095979 14188 0 788095981 14189 0 788095982 14190 0 788095984 14191 0 788095984 14192 0 788095986 14193 0 788095987 14194 0 788095989 14195 0 788095997 14196 0 788095998 14197 0 788096000 14198 0 788096002 14199 0 788096004 14200 0 788096004 14201 0 788096006 14202 0 788096015 14203 0 788096017 14204 0 788096018 14205 0 788096019 14206 0 788096021 14207 0 788096023 14208 0 788096023 14209 0 788096031 14210 0 788096034 14211 0 788096035 14212 0 788096037 14213 0 788096038 14214 0 788096040 14215 0 788096048 14216 0 788096050 14217 0 788096050 14218 0 788096052 14219 0 788096053 14220 0 788096055 14221 0 788096057 14222 0 788096065 14223 0 788096068 14224 0 788096071 14225 0 788096072 14226 0 788096083 14227 0 788096088 14228 0 788096101 14229 0 788096114 14230 0 788096120 14231 0 788096134 14232 0 788096147 14233 0 788096160 14234 0 788096161 14235 0 788096174 14236 0 788096179 14237 0 788096191 14238 0 788096203 14239 0 788096209 14240 0 788096220 14241 0 788096225 14242 0 788096225 14243 0 788096236 14244 0 788096240 14245 0 788096251 14246 0 788096255 14247 0 788096265 14248 0 788096269 14249 0 788096273 14250 0 788096284 14251 0 788096284 14252 0 788096297 14253 0 788096303 14254 0 788096317 14255 0 788096329 14256 0 788096333 14257 0 788096343 14258 0 788096346 14259 0 788096347 14260 0 788096349 14261 0 788096351 14262 0 788096360 14263 0 788096363 14264 0 788096365 14265 0 788096366 14266 0 788096369 14267 0 788096377 14268 0 788096377 14269 0 788096379 14270 0 788096382 14271 0 788096385 14272 0 788096394 14273 0 788096398 14274 0 788096401 14275 0 788096411 14276 0 788096412 14277 0 788096416 14278 0 788096428 14279 0 788096444 14280 0 788096451 14281 0 788096463 14282 0 788096475 14283 0 788096481 14284 0 788096492 14285 0 788096493 14286 0 788096504 14287 0 788096509 14288 0 788096520 14289 0 788096525 14290 0 788096536 14291 0 788096538 14292 0 788096547 14293 0 788096547 14294 0 788096551 14295 0 788096562 14296 0 788096566 14297 0 788096569 14298 0 788096578 14299 0 788096581 14300 0 788096583 14301 0 788096585 14302 0 788096586 14303 0 788096594 14304 0 788096596 14305 0 788096598 14306 0 788096599 14307 0 788096601 14308 0 788096609 14309 0 788096611 14310 0 788096611 14311 0 788096613 14312 0 788096614 14313 0 788096615 14314 0 788096617 14315 0 788096618 14316 0 788096627 14317 0 788096628 14318 0 788096630 14319 0 788096630 14320 0 788096631 14321 0 788096633 14322 0 788096634 14323 0 788096635 14324 0 788096637 14325 0 788096646 14326 0 788096647 14327 0 788096647 14328 0 788096649 14329 0 788096650 14330 0 788096652 14331 0 788096653 14332 0 788096662 14333 0 788096663 14334 0 788096664 14335 0 788096666 14336 0 788096666 14337 0 788096668 14338 0 788096669 14339 0 788096671 14340 0 788096679 14341 0 788096681 14342 0 788096682 14343 0 788096683 14344 0 788096683 14345 0 788096686 14346 0 788096688 14347 0 788096697 14348 0 788096698 14349 0 788096699 14350 0 788096701 14351 0 788096702 14352 0 788096704 14353 0 788096704 14354 0 788096706 14355 0 788096714 14356 0 788096715 14357 0 788096717 14358 0 788096721 14359 0 788096731 14360 0 788096734 14361 0 788096735 14362 0 788096737 14363 0 788096739 14364 0 788096747 14365 0 788096749 14366 2 788096749 2024-12-22T00:19:01Z INFO __main__ 222: generated L0 files: [PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784144540-0784166144_V02_2411055007-50670.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0783785317-0783788044_V02_2411010773-50671.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784101340-0784122945_V02_2411052192-50668.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784122946-0784144540_V02_2411054602-50669.fits')] 2024-12-22T00:19:02Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20241213_002.tm 2024-12-22T00:19:02Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20241221_002.tm 2024-12-22T00:19:02Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784144540-0784166144_V02_2411055007-50670.fits 2024-12-22T00:19:02Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-12-22T00:19:03Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0783785317-0783788044_V02_2411010773-50671.fits 2024-12-22T00:19:03Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-12-22T00:19:03Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784101340-0784122945_V02_2411052192-50668.fits 2024-12-22T00:19:03Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-12-22T00:19:04Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0784122946-0784144540_V02_2411054602-50669.fits 2024-12-22T00:19:04Z INFO stixcore.soop.manager 484: Soop found: SOOP 2024-12-22T00:19:04Z INFO __main__ 226: generated L1 files: [PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/11/05/SCI/solo_L1_stix-sci-xray-cpd_20241105T055959-20241105T120005_V02_2411052192-50668.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/11/01/SCI/solo_L1_stix-sci-xray-cpd_20241101T141255-20241101T145822_V02_2411010773-50671.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/11/05/SCI/solo_L1_stix-sci-xray-cpd_20241105T120005-20241105T180000_V02_2411054602-50669.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/11/05/SCI/solo_L1_stix-sci-xray-cpd_20241105T180000-20241106T000004_V02_2411055007-50670.fits')]