2024-05-11T12:32:50Z INFO __main__ 304: PIPELINE VERSION Version: 1.4.1 Common instrument config version: v0.1.4 PROCESSING VERSIONS Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-05-11T12:32:50Z INFO __main__ 269: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1 waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-05-11T12:32:50Z INFO __main__ 300: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 32758 SPICE: [PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240425_004.tm'), PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240510_001.tm')] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.37', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.37', 'version': ['2', '26', '37']}, {'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-05-11T12:32:51Z INFO stixcore.io.RidLutManager 154: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20240425/20240525 2024-05-11T12:32:51Z INFO stixcore.io.RidLutManager 163: found 2 entries 2024-05-11T12:32:52Z INFO stixcore.io.RidLutManager 172: write total 32758 entries to local storage 2024-05-11T12:32:53Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_hourly_req_BatchRequest.PktTmRaw.SOL.0.2024.106.14.15.29.426.NpDz@2024.132.10.00.01.119.2.xml') 2024-05-11T12:32:54Z INFO stixcore.io.fits.processors 444: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403277504-61983.fits exists appending data 2024-05-11T12:32:54Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403277504-61983.fits 2024-05-11T12:32:54Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403272157-61985.fits 2024-05-11T12:32:54Z INFO stixcore.io.fits.processors 475: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403274286-61984.fits 2024-05-11T12:32:54Z INFO __main__ 223: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403272157-61985.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403274286-61984.fits'), PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403277504-61983.fits')} 2024-05-11T12:32:55Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-05-11T12:32:55Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240425_004.tm 2024-05-11T12:32:55Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240510_001.tm 2024-05-11T12:32:55Z INFO stixcore.io.RidLutManager 174: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-05-11T12:33:22Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 15615 1 768523920 15616 0 768523925 15617 0 768523926 15618 0 768523931 15619 0 768523933 15620 0 768523938 15621 0 768523944 15622 0 768523946 15623 0 768523946 15624 0 768523950 15625 0 768523952 15626 0 768523956 15627 0 768523962 15628 0 768523969 15629 0 768523977 15630 0 768523982 15631 0 768523984 15632 0 768523984 15633 0 768523991 15634 0 768523992 15635 0 768523994 15636 0 768523998 15637 0 768523999 15638 0 768524000 15639 0 768524004 15640 0 768524004 15641 0 768524005 15642 0 768524006 15643 0 768524007 15644 0 768524011 15645 0 768524013 15646 0 768524017 15647 0 768524018 15648 0 768524019 15649 0 768524020 15650 0 768524024 15651 0 768524025 15652 0 768524027 15653 0 768524030 15654 0 768524032 15655 0 768524033 15656 0 768524037 15657 0 768524037 15658 0 768524038 15659 0 768524040 15660 0 768524044 15661 0 768524049 15662 0 768524052 15663 0 768524057 15664 0 768524059 15665 0 768524063 15666 0 768524063 15667 0 768524065 15668 0 768524070 15669 0 768524075 15670 0 768524077 15671 0 768524081 15672 0 768524084 15673 0 768524089 15674 0 768524089 15675 0 768524095 15676 0 768524097 15677 0 768524102 15678 0 768524104 15679 0 768524108 15680 0 768524109 15681 0 768524110 15682 0 768524114 15683 0 768524115 15684 0 768524116 15685 0 768524123 15686 0 768524125 15687 0 768524132 15688 0 768524145 15689 0 768524156 15690 0 768524161 15691 0 768524161 15692 0 768524163 15693 0 768524164 15694 0 768524168 15695 0 768524170 15696 0 768524175 15697 0 768524177 15698 0 768524182 15699 0 768524188 15700 0 768524188 15701 0 768524190 15702 0 768524195 15703 0 768524200 15704 0 768524203 15705 0 768524208 15706 0 768524213 15707 0 768524214 15708 0 768524214 15709 0 768524216 15710 0 768524220 15711 0 768524221 15712 0 768524226 15713 0 768524229 15714 0 768524234 15715 0 768524236 15716 0 768524240 15717 0 768524241 15718 0 768524242 15719 0 768524246 15720 0 768524248 15721 0 768524253 15722 0 768524255 15723 0 768524260 15724 0 768524262 15725 0 768524262 15726 0 768524267 15727 0 768524269 15728 0 768524273 15729 0 768524276 15730 0 768524281 15731 0 768524286 15732 0 768524293 15733 0 768524296 15734 0 768524296 15735 0 768524302 15736 0 768524320 15737 0 768524322 15738 0 768524327 15739 0 768524329 15740 0 768524333 15741 0 768524334 15742 0 768524334 15743 0 768524336 15744 0 768524349 15745 0 768524378 15746 0 768524390 15747 0 768524392 15748 0 768524396 15749 0 768524397 15750 0 768524398 15751 0 768524398 15752 0 768524400 15753 0 768524404 15754 0 768524408 15755 0 768524409 15756 0 768524414 15757 0 768524415 15758 0 768524416 15759 0 768524416 15760 0 768524418 15761 0 768524422 15762 0 768524423 15763 0 768524424 15764 0 768524425 15765 0 768524431 15766 0 768524468 15767 0 768524512 15768 0 768524513 15769 0 768524549 15770 0 768524585 15771 0 768524620 15772 0 768524653 15773 0 768524682 15774 0 768524711 15775 0 768524734 15776 0 768524734 15777 0 768524754 15778 0 768524773 15779 0 768524794 15780 0 768524808 15781 0 768524820 15782 0 768524832 15783 0 768524840 15784 0 768524850 15785 0 768524851 15786 0 768524858 15787 0 768524864 15788 0 768524874 15789 0 768524877 15790 0 768524883 15791 0 768524885 15792 0 768524889 15793 0 768524890 15794 0 768524892 15795 0 768524898 15796 0 768524901 15797 0 768524902 15798 0 768524908 15799 0 768524910 15800 0 768524916 15801 0 768524924 15802 0 768524925 15803 0 768524927 15804 0 768524932 15805 0 768524934 15806 0 768524939 15807 0 768524941 15808 0 768524950 15809 0 768524956 15810 0 768524956 15811 0 768524958 15812 0 768524963 15813 0 768524965 15814 0 768524966 15815 0 768524971 15816 0 768524973 15817 0 768524980 15818 0 768524983 15819 0 768524983 15820 0 768524989 2024-05-11T12:33:22Z INFO __main__ 227: generated L0 files: [PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0764834177-0764855776_V02_2403277504-61983.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0764855776-0764877376_V02_2403274286-61984.fits')] 2024-05-11T12:33:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240425_004.tm 2024-05-11T12:33:22Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240510_001.tm 2024-05-11T12:33:22Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0764834177-0764855776_V02_2403277504-61983.fits 2024-05-11T12:33:22Z INFO stixcore.soop.manager 482: No soops found for time: 2024-03-27 06:00:01.466454 - 2024-03-27 12:00:00.806522 2024-05-11T12:33:23Z INFO stixcore.processing.L0toL1 90: processing file: /data/stix/out/fits_v1.2.0/L0/21/6/21/solo_L0_stix-sci-xray-cpd_0764855776-0764877376_V02_2403274286-61984.fits 2024-05-11T12:33:23Z INFO stixcore.soop.manager 482: No soops found for time: 2024-03-27 12:00:00.806537 - 2024-03-27 18:00:00.046614 2024-05-11T12:33:23Z INFO __main__ 231: generated L1 files: [PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/03/27/SCI/solo_L1_stix-sci-xray-cpd_20240327T120000-20240327T180000_V02_2403274286-61984.fits'), PosixPath('/data/stix/out/fits_v1.2.0/L1/2024/03/27/SCI/solo_L1_stix-sci-xray-cpd_20240327T060001-20240327T120000_V02_2403277504-61983.fits')] 2024-05-11T12:33:23Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20240425_004.tm 2024-05-11T12:33:24Z INFO stixcore.ephemeris.manager 179: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20240510_001.tm 2024-05-11T12:33:24Z WARNING stixcore.processing.L1toL2 99: No L2 product match for product