CCS fails the check for overwriting files

[jpinedaf]

When calibrating data using the doPipeline function, CCS fails the check to decide if the files should be overwritten.

The problem is that CCS is observed in a single feed, while the check requires that all feeds should be reduced to skip them. A similar issue occurs should occur when a Vegas bank is missing from the observations. The relevant piece of code is here:

       for bank in BankNames:
        # First check to see if a pipeline call is necessary. 
        FilesIntact = True
        if not overwrite:
            for feed in ['0','1','2','3','4','5','6']:
                for pol in ['0','1']:
                    outputfile = Source+'_scan_{0}_{1}_window{2}_feed{3}_pol{4}_sess{5}.fits'.\
                        format(StartScan,EndScan,Window,feed,pol,SessionNumber) 
                    FilesIntact = FilesIntact and os.path.exists(OutputDirectory+'/'+outputfile)
                    if FilesIntact:
                        print('Data for Polarization {0} of Feed {1} appear on disk... skipping'.format(pol,feed))
        if (not FilesIntact) or (overwrite):
            InputFile = RawDataDir+SessionDir+'AGBT15A_430_'+\
                str(SessionNumber).zfill(2)+\
                '.raw.vegas.{0}.fits'.format(bank)

A possible solution would be to do the check for each Bank independently. Am I missing something @rfriesen @low-sky ?

[jpinedaf]

I checked, and this solution seems to solve the issue, however, OrionA seems to keep having problems on moving and renaming files. Really strange!

If you check the files present in /lustre/pipeline/scratch/GAS/OrionA you will find several files that should have been moved into the /lustre/pipeline/scratch/GAS/OrionA/OrionA_NH3_11 folder. However, when I run the same code used in the pipeline manually, then the files are moved, although I do get an exception from the so.rename() call. I am stumped!