2024-12-04T11:33:09Z INFO __main__ 305: PIPELINE VERSION Version: 1.5.0 Common instrument config version: v0.1.5 PROCESSING VERSIONS Prod: Ephemeris File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/ANC/aspect.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/housekeepingL0.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/quicklookL0.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level0/scienceL0.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/housekeepingL1.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/quicklookL1.py Vers: 2 Prod: RawPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: CompressedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: SummedPixelData File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Visibility File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Spectrogram File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: Aspect File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level1/scienceL1.py Vers: 2 Prod: MiniReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: MaxiReport File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/housekeepingL2.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Background File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Spectra File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: Variance File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: EnergyCalibration File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: TMStatusFlareList File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/level2/quicklookL2.py Vers: 2 Prod: LevelB File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/levelb/binary.py Vers: 2 Prod: LightCurve File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 Prod: FlareFlag File: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/products/lowlatency/quicklookLL.py Vers: 2 2024-12-04T11:33:09Z INFO __main__ 270: CONFIG [Paths] tm_archive = /data/stix/SOLSOC/from_edds/tm/processing fits_archive = /data/stix/out/fits_v1.2.0 spice_kernels = /data/stix/spice/kernels soop_files = /data/stix/SOLSOC/from_soc [Logging] stop_on_error = False [IDLBridge] enabled = True gsw_path = /opt/STIX-GSW batchsize = 30 [Pipeline] status_server_port = 12388 parallel_batchsize_l0 = 100 parallel_batchsize_l1 = 100 parallel_batchsize_l2 = 100 log_level = INFO log_dir = /data/stix/out/fits_v1.2.0/logging error_mail_send = True error_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie error_mail_smpt_host = lmailer.fhnw.ch error_mail_smpt_port = 25 error_mail_sender = nicky.hochmuth@fhnw.ch start_with_unprocessed = True history_db_file = /home/stixcore/soar_pub/processing_history.sqlite [Publish] db_file = /home/stixcore/soar_pub/published.sqlite include_levels = L0, L1, ANC waiting_period = 21d target_dir = /home/stixcore/from_pub099 report_mail_receivers = nicky.hochmuth@ateleris.ch, shane.maloney@dias.ie report_mail_send = True same_esa_name_dir = /data/stix/out/esa_conflicts rid_lut_file = ./stixcore/data/publish/rid_lut.csv rid_lut_file_update_url = https://datacenter.stix.i4ds.net/api/bsd/info/ batch_size = 20000 fits_dir = /data/stix/out/fits_v1.2.0 target_host = pub026.cs.technik.fhnw.ch supplement_report = /home/stixcore/soar_pub/supplement_report.csv include_products = ql,hk,sci,cal log_dir = /data/stix/out/fits_v1.2.0/logging [SOOP] endpoint = https://solarorbiter.esac.esa.int/soopkitchen/api user = smaloney password = xxxx soop_files_download = ./stixcore/data/soop 2024-12-04T11:33:09Z INFO __main__ 301: SINGLETONS SOOPManager: /data/stix/SOLSOC/from_soc RidLutManager: file: stixcore/data/publish/rid_lut.csv update: True size: 33663 SPICE: [(PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20241127_004.tm'), 'pred', datetime.datetime(2024, 11, 27, 0, 0)), (PosixPath('/data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20241204_001.tm'), 'flown', datetime.datetime(2024, 12, 4, 0, 0))] IDBManager: /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb Versions: [{'label': '2.26.36', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.36', 'version': ['2', '26', '36']}, {'label': '2.26.32', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.32', 'version': ['2', '26', '32']}, {'label': '2.26.34', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.34', 'version': ['2', '26', '34']}, {'label': '2.26.38', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38', 'version': ['2', '26', '38']}, {'label': '2.26.33', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.33', 'version': ['2', '26', '33']}, {'label': '2.26.31', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.31', 'version': ['2', '26', '31']}, {'label': '2.26.35', 'path': '/home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.35', 'version': ['2', '26', '35']}] Force version: None History: IntervalTree([Interval(0.0, 640198039.0, '2.26.32'), Interval(640198039.0, 640265396.0, '2.26.31'), Interval(640265396.0, 642159993.0, '2.26.32'), Interval(642159993.0, 643123725.0, '2.26.33'), Interval(643123725.0, 677774249.0, '2.26.34'), Interval(677774249.0, 692380776.0, '2.26.35'), Interval(692380776.0, 761270182.0, '2.26.36'), Interval(761270182.0, 4294967295.0, '2.26.38')]) 2024-12-04T11:33:10Z INFO stixcore.io.RidLutManager 158: download publish lut file: https://datacenter.stix.i4ds.net/api/bsd/info/20241124/20241224 2024-12-04T11:33:10Z INFO stixcore.io.RidLutManager 167: found 2 entries 2024-12-04T11:33:11Z INFO stixcore.io.RidLutManager 176: write total 33663 entries to local storage 2024-12-04T11:33:12Z INFO __main__ 495: Opened file: /data/stix/SOLSOC/from_edds/tm/processing/TM_Manual_Req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.47.27.724.kwBf@2024.339.09.47.29.182.2.xml 2024-12-04T11:33:12Z INFO __main__ 490: Closed read file: /data/stix/SOLSOC/from_edds/tm/processing/TM_Manual_Req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.47.27.724.kwBf@2024.339.09.47.29.182.2.xml 2024-12-04T11:33:12Z INFO stixcore.processing.TMTCtoLB 30: Processing file: SOCPacketFile('/data/stix/SOLSOC/from_edds/tm/processing/TM_Manual_Req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.47.27.724.kwBf@2024.339.09.47.29.182.2.xml') 2024-12-04T11:33:12Z INFO __main__ 495: Opened file: /data/stix/SOLSOC/from_edds/tm/processing/TM_Manual_Req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.47.27.724.kwBf@2024.339.09.47.29.182.2.xml 2024-12-04T11:33:12Z INFO __main__ 490: Closed read file: /data/stix/SOLSOC/from_edds/tm/processing/TM_Manual_Req_BatchRequest.PktTmRaw.SOL.0.2024.339.09.47.27.724.kwBf@2024.339.09.47.29.182.2.xml 2024-12-04T11:33:12Z INFO stixcore.io.fits.processors 448: Fits file solo_LB_stix-21-6-21_0000000000-9999999999_V02_2410051300-49457.fits exists appending data 2024-12-04T11:33:12Z INFO stixcore.io.fits.processors 479: Writing fits file to /data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2410051300-49457.fits 2024-12-04T11:33:12Z INFO __main__ 224: generated LB files: {PosixPath('/data/stix/out/fits_v1.2.0/LB/21/6/21/solo_LB_stix-21-6-21_0000000000-9999999999_V02_2410051300-49457.fits')} 2024-12-04T11:33:12Z INFO stixcore.idb.idb 522: IDB loaded from /home/stixcore/STIXCore/venv/lib/python3.9/site-packages/stixcore/data/idb/v2.26.38/idb.sqlite 2024-12-04T11:33:12Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-pred-mk_V111_20241127_004.tm 2024-12-04T11:33:12Z INFO stixcore.ephemeris.manager 198: LOADING NEW META KERNEL: /data/stix/spice/kernels/mk/solo_ANC_soc-flown-mk_V111_20241204_001.tm 2024-12-04T11:33:12Z INFO stixcore.io.RidLutManager 178: read rid-lut from stixcore/data/publish/rid_lut.csv 2024-12-04T11:33:14Z WARNING stixcore.products.levelb.binary 322: Incomplete sequence sequence_count sequence_flag scet_coarse -------------- ------------- ----------- 12616 1 786585786 12617 0 786585787 12618 0 786585789 12619 0 786585790 12620 0 786585791 12621 0 786585792 12622 0 786585800 12623 0 786585801 12624 0 786585802 12625 0 786585803 12626 0 786585804 12627 0 786585805 12628 0 786585806 12629 0 786585807 12630 0 786585808 12631 0 786585809 12632 0 786585811 12633 0 786585811 12634 0 786585819 12635 0 786585822 12636 0 786585825 12637 0 786585828 12638 0 786585838 12639 0 786585841 12640 0 786585845 12641 0 786585845 12642 0 786585855 12643 0 786585858 12644 0 786585861 12645 0 786585870 12646 0 786585873 12647 0 786585875 12648 0 786585877 12649 0 786585879 12650 0 786585879 12651 0 786585887 12652 0 786585889 12653 0 786585891 12654 0 786585892 12655 0 786585894 12656 0 786585895 12657 0 786585896 12658 0 786585897 12659 0 786585898 12660 0 786585905 12661 0 786585906 12662 0 786585907 12663 0 786585909 12664 0 786585910 12665 0 786585910 12666 0 786585911 12667 0 786585912 12668 0 786585913 12669 0 786585914 12670 0 786585915 12671 0 786585916 12672 0 786585923 12673 0 786585925 12674 0 786585926 12675 0 786585926 12676 0 786585927 12677 0 786585929 12678 0 786585930 12679 0 786585931 12680 0 786585932 12681 0 786585933 12682 0 786585934 12683 0 786585935 12684 0 786585936 12685 0 786585943 12686 0 786585944 12687 0 786585945 12688 0 786585946 12689 0 786585947 12690 0 786585949 12691 0 786585950 12692 0 786585951 12693 0 786585952 12694 0 786585954 12695 0 786585955 12696 0 786585963 12697 0 786585964 12698 0 786585965 12699 0 786585966 12700 0 786585967 12701 0 786585968 12702 0 786585969 12703 0 786585970 12704 0 786585971 12705 0 786585973 12706 0 786585974 12707 0 786585975 12708 0 786585986 12709 0 786585987 12710 0 786585989 12711 0 786585990 12712 0 786585992 12713 0 786585994 12714 0 786585996 12715 0 786586005 12716 0 786586006 12717 0 786586007 12718 0 786586008 12719 0 786586010 12720 0 786586011 12721 0 786586013 12722 0 786586014 12723 0 786586015 12724 0 786586016 12725 0 786586025 12726 0 786586025 12727 0 786586026 12728 0 786586027 12729 0 786586029 12730 0 786586030 12731 0 786586033 12732 0 786586034 12733 0 786586036 12734 0 786586045 12735 0 786586045 12736 0 786586047 12737 0 786586049 12738 0 786586051 12739 0 786586052 12740 0 786586054 12741 0 786586056 12742 0 786586065 12743 0 786586066 12744 0 786586069 12745 0 786586072 12746 0 786586082 12747 0 786586085 12748 0 786586087 12749 0 786586090 12750 0 786586099 12751 0 786586101 12752 0 786586101 12753 0 786586103 12754 0 786586105 12755 0 786586107 12756 0 786586117 12757 0 786586118 12758 0 786586121 12759 0 786586123 12760 0 786586123 12761 0 786586126 12762 0 786586134 12763 0 786586135 12764 0 786586138 12765 0 786586140 12766 0 786586142 12767 0 786586151 12768 0 786586153 12769 0 786586153 12770 0 786586156 12771 0 786586158 12772 0 786586161 12773 0 786586169 12774 0 786586172 12775 0 786586174 12776 0 786586177 12777 0 786586177 12778 0 786586185 12779 0 786586187 12780 0 786586190 12781 0 786586193 12782 0 786586202 12783 0 786586205 12784 0 786586207 12785 0 786586217 12786 0 786586217 12787 0 786586220 12788 0 786586224 12789 0 786586234 12790 0 786586237 12791 0 786586247 12792 0 786586251 12793 0 786586262 12794 0 786586262 12795 0 786586267 12796 0 786586277 12797 0 786586282 12798 0 786586291 12799 0 786586302 12800 0 786586306 12801 0 786586317 12802 0 786586321 12803 0 786586322 12804 0 786586331 12805 0 786586336 12806 0 786586346 12807 0 786586352 12808 0 786586363 12809 0 786586367 12810 0 786586376 12811 0 786586377 12812 0 786586381 12813 0 786586390 12814 0 786586394 12815 0 786586404 12816 0 786586408 2024-12-04T11:33:14Z INFO __main__ 228: generated L0 files: [] 2024-12-04T11:33:14Z INFO __main__ 232: generated L1 files: [] 2024-12-04T11:33:14Z INFO __main__ 236: generated L2 files: []