Multiple resources failing to return correct result on test #500
Comments
|
@modfh98 , I think this relates to the latest Security Updates (SU) changes. In the last one posted here: Microsoft has enabled Powershell Serialization by default. These have brought some known issues that are listed here: We utilize this Powershell module quite often. Do you also? We could try to see if we can modify the code so it's compatible with Powershell serialization. What do you think? also, do you have any suggestions @johlju ? |
|
@mhincapie |
modfh98
changed the title
|
Awesome that you help each other finding the cause, I wouldn't have know about this, thanks @mhincapie. If you see the fix from MS i dragging out I'm happy to review a PR that uses the suggested workaround. @modfh98 maybe you can change the code, according to the suggested workaround, locally for one of the resources (suggest take the simplest one) on one of your nodes just to verify that it actually quick-fixes the problem? |
|
Did anyone have any success implementing a workaround for this? I spent several hours troubleshooting the ExchExchangeCertificate resource with DSC Debug enabled but wasn't able to make any headway. I could successfully convert this object into the standard certificate type: But I wasn't able to convert it back into its correct type: Thus, the object was missing the "Services" property was what was causing the Test-TargetResource to fail. Like @modfh98, I've had issues with some of the resources they referenced including:
I haven't done any significant troubleshooting for these so it's possible there's a simpler workaround for these. When I disabled certificate signing of PowerShell serialization payloads using the method below, this problem disappeared. We enabled certificate signing of PowerShell serialization payloads back in February 2023 and it's been a problem ever since. I would have thought Microsoft would have invested some more time in maintaining this resource. |
|
I have the same concern, was expecting MSFT fix the serialization issue by now. Is there any timeline for the SU issue fix, does anybody know? thanks, |
|
Do you guys know if there is any MSFT internal discussion how the DSC issue caused by ps payload siging will be solved or will be solved at all? |
|
I am trying to get into this a bit more and need a very simple configuration to start with. So far I cannot reproduce the issue. At the very top it is mentioned that the issue affects the resource configuration c1 {
Import-DscResource -ModuleName ExchangeDsc
$cred = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
node localhost {
ExchReceiveConnector rc1 {
Identity = 'MSEEx1\rc1'
Credential = $cred
Ensure = 'Present'
Bindings = '0.0.0.0:2525'
RemoteIPRanges = '192.168.0.1-192.168.0.24'
Usage = 'Custom'
}
}
}
$cd = @{
AllNodes = @(
@{
NodeName = 'localhost'
PSDscAllowPlainTextPassword = $true
}
)
}
c1 -OutputPath c:\dsc -ConfigurationData $cd
Start-DscConfiguration -Path C:\dsc -Wait -Verbose -ForceGet-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument RefreshWhat do I need to do to run into the issue? |
|
@modfh98, is this issue also happening on Exchange 2019 CU14 or only with Exchange 2016? On Exchange 2019 CU14 everything works just fine for me. |
|
@johlju, can you flag this issue as a bug, please? |
|
I am not an expert in Exchange, so I cannot comment on the cause. This is why for example the The object type of some values has been changed, probably because of what @mhincapie explained above. In PathName property which does not exist anymore as $db.LogFolderPath is a string.
The same happens here:
The solution would be to change the code so that is uses the value stored in the property @gborus, can you give it a try and change these 3 lines, please? Given the fact that there has been no feedback so far, is ExchangeDsc still being used at all? |
johlju
added
bug
|
Hi @raandree , the suggested solution would solve this particular problem, although this would mean all resources' script would need to be review line by line to spot similar problems, not mentioning cases like the DB copies' activation preference query which data is available at the 3rd level, like database.databasecopies.activation preference, which data gets lost when database.databasecopies gets converted to string. And the trick with this value, activation preference, that it is really hard to collect it from elsewhere other than the get-mailboxdatabase cmdlet. thanks, |
|
I have isolated the code that creates the connection to Exchange in the module The script to reproduce the issue is this the following. The code is taken from the function Invoke-DotSourcedScript {
[CmdletBinding()]
[OutputType([System.Collections.Hashtable])]
param
(
[Parameter(Mandatory = $true)]
[System.String]
$ScriptPath,
[Parameter()]
[System.Collections.Hashtable]
$ScriptParams = @{ },
[Parameter()]
[System.String[]]
$SnapinsToRemove,
[Parameter()]
[System.String[]]
$CommandsToExecuteInScope,
[Parameter()]
[System.Collections.Hashtable]
$ParamsForCommandsToExecuteInScope
)
[System.Collections.Hashtable] $returnValues = @{ }
. $ScriptPath @ScriptParams
for ($i = 0; $i -lt $CommandsToExecuteInScope.Count; $i++) {
[System.String] $commandToExecute = $CommandsToExecuteInScope[$i]
[System.Collections.Hashtable] $commandParams = @{ }
if ($ParamsForCommandsToExecuteInScope.ContainsKey($commandToExecute)) {
[System.Collections.Hashtable] $commandParams = $ParamsForCommandsToExecuteInScope[$commandToExecute]
}
$returnValue = . $commandToExecute @commandParams
if (!$returnValues.ContainsKey($commandToExecute)) {
$returnValues.Add($commandToExecute, $null)
}
$returnValues[$commandToExecute] = $returnValue
}
if ($SnapinsToRemove.Count -gt 0) {
Remove-HelperSnapin -SnapinsToRemove $SnapinsToRemove -Verbose:$VerbosePreference
}
return $returnValues
}
$DSCExchangeModulePath = 'C:\Temp'
$Credential = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
$serverFQDN = 'MSEEx1.contoso.com'
$exbin = Join-Path -Path ((Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\Setup).MsiInstallPath) -ChildPath 'bin'
$remoteExchange = Join-Path -Path "$exbin" -ChildPath 'RemoteExchange.ps1'
$commandToExecuteAfterDotSourcing = @('_NewExchangeRunspace')
$commandParamsToExecuteAfterDotSourcing = @{
'_NewExchangeRunspace' = @{
fqdn = $serverFQDN
credential = $Credential
UseWIA = $false
AllowRedirection = $false
}
}
$returnValues = Invoke-DotSourcedScript `
-ScriptPath $remoteExchange `
-CommandsToExecuteInScope $commandToExecuteAfterDotSourcing `
-ParamsForCommandsToExecuteInScope $commandParamsToExecuteAfterDotSourcing `
-Verbose:$VerbosePreference
if ($null -ne $returnValues -and $returnValues.ContainsKey('_NewExchangeRunspace')) {
$session = $returnValues['_NewExchangeRunspace']
}
if ($null -ne $session) {
$session.Name = 'DSCExchangeSession'
}
New-Item -Path $DSCExchangeModulePath -Type Directory -Force | Out-Null
Export-PSSession -Session $Session -OutputModule $DSCExchangeModulePath -Force -AllowClobber | Out-Null
Import-Module -Name $DSCExchangeModulePath -Global -DisableNameChecking -Function *
$dbs = Get-MailboxDatabase
$dbs[0].LogFolderPathThe serialized data is in |
|
Just adding some thoughts, I don't have a clue what could be the difference between running it locally and in DSC. If it is related to certificate signing of PowerShell serialization payloads as mentioned in an above comment could it be that the user the resource runs at does not have the necessary user rights or permission? Or could it be that the session that is exported is using a different account so the signing is invalid for some reason. 🤔 |
|
Thank, @johlju. I am not sure which certificate is used. There are no certificates in the user's scope but some in the computer's one. The output of the certificates is the same in a working and non-working process: |
|
It says in the article https://learn.microsoft.com/en-us/exchange/plan-and-deploy/post-installation-tasks/security-best-practices/exchange-serialization-payload-sign?view=exchserver-2019 that the certificate is "Exchange Server Auth Certificate"
Is it possible to disable signing just to confirm that it is actually signing that is the issue: https://learn.microsoft.com/en-us/exchange/plan-and-deploy/post-installation-tasks/security-best-practices/exchange-serialization-payload-sign?view=exchserver-2019#disable-certificate-signing-of-powershell-serialization-payloads |
|
Also, have you tried the code you tested locally above in a DSC resource too so that there aren't anything in the actual DSC resource that fails. In the article above it states that piping to commands could fail too. So maybe try to minimize the code so few lines as possible in a DSC resource to see if it starts to work? 🤔 |
Thanks. The DSC LCM process can access the And yes, the customer and we made sure that the issue only exists when signing is enabled. |
You mean something like this? The working code I have shared above fails when it runs in the LCM context. Configuration MyDscConfiguration {
param
(
[PSCredential]$Creds
)
Import-DscResource -Module ExchangeDsc
Node MSEEx1 {
Script SetupExchangeConnect
{
GetScript = {
return @{ Result = '' }
}
TestScript = {
return $false
}
SetScript = {
function Invoke-DotSourcedScript {
[CmdletBinding()]
[OutputType([System.Collections.Hashtable])]
param
(
[Parameter(Mandatory = $true)]
[System.String]
$ScriptPath,
[Parameter()]
[System.Collections.Hashtable]
$ScriptParams = @{ },
[Parameter()]
[System.String[]]
$SnapinsToRemove,
[Parameter()]
[System.String[]]
$CommandsToExecuteInScope,
[Parameter()]
[System.Collections.Hashtable]
$ParamsForCommandsToExecuteInScope
)
[System.Collections.Hashtable] $returnValues = @{ }
. $ScriptPath @ScriptParams
for ($i = 0; $i -lt $CommandsToExecuteInScope.Count; $i++) {
[System.String] $commandToExecute = $CommandsToExecuteInScope[$i]
[System.Collections.Hashtable] $commandParams = @{ }
if ($ParamsForCommandsToExecuteInScope.ContainsKey($commandToExecute)) {
[System.Collections.Hashtable] $commandParams = $ParamsForCommandsToExecuteInScope[$commandToExecute]
}
$returnValue = . $commandToExecute @commandParams
if (!$returnValues.ContainsKey($commandToExecute)) {
$returnValues.Add($commandToExecute, $null)
}
$returnValues[$commandToExecute] = $returnValue
}
if ($SnapinsToRemove.Count -gt 0) {
Remove-HelperSnapin -SnapinsToRemove $SnapinsToRemove -Verbose:$VerbosePreference
}
return $returnValues
}
$DSCExchangeModulePath = 'C:\Temp'
$Credential = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
$serverFQDN = 'MSEEx1.contoso.com'
$exbin = Join-Path -Path ((Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\Setup).MsiInstallPath) -ChildPath 'bin'
$remoteExchange = Join-Path -Path "$exbin" -ChildPath 'RemoteExchange.ps1'
$commandToExecuteAfterDotSourcing = @('_NewExchangeRunspace')
$commandParamsToExecuteAfterDotSourcing = @{
'_NewExchangeRunspace' = @{
fqdn = $serverFQDN
credential = $Credential
UseWIA = $false
AllowRedirection = $false
}
}
$returnValues = Invoke-DotSourcedScript `
-ScriptPath $remoteExchange `
-CommandsToExecuteInScope $commandToExecuteAfterDotSourcing `
-ParamsForCommandsToExecuteInScope $commandParamsToExecuteAfterDotSourcing `
-Verbose:$VerbosePreference
if ($null -ne $returnValues -and $returnValues.ContainsKey('_NewExchangeRunspace')) {
$session = $returnValues['_NewExchangeRunspace']
}
if ($null -ne $session) {
$session.Name = 'DSCExchangeSession'
}
New-Item -Path $DSCExchangeModulePath -Type Directory -Force | Out-Null
Export-PSSession -Session $Session -OutputModule $DSCExchangeModulePath -Force -AllowClobber | Out-Null
Import-Module -Name $DSCExchangeModulePath -Global -DisableNameChecking -Function *
Wait-Debugger
$dbs = Get-MailboxDatabase
$dbs[0].LogFolderPath
}
}
}
}
$HostHash = @{
AllNodes = @(
@{
NodeName = "$env:COMPUTERNAME"
PSDscAllowPlainTextPassword = $true
PSDscAllowDomainUser = $true
};
);
}
mkdir "C:\ExConfig" -ErrorAction SilentlyContinue
$myCred = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
MyDscConfiguration -ConfigurationData $HostHash -Creds $mycred -OutputPath C:\ExConfig
Start-DscConfiguration C:\ExConfig -Force -Wait -Verbose |
we disabled payload signing and that solved the problem, otherwise we couldn't get DSC working... |
|
@raandree What happens if you run the resource as CONTOSO\install as well using property PsDscRunAsCredential? Then the resource runs as the same user as used inside the resource 🤔 |
|
Thanks for the hint, @johlju. That did the trick and things are working again. However, I have no idea why it is necessary to use alternative credentials as the LCM should have access to all relevant resources including the certificates and their private keys. If anyone can explain why using @gborus, can you please test this in your environment as well? This configuration works for me: configuration ExchangeMailboxConfig {
param
(
[PSCredential]$Creds
)
Import-DscResource -Module ExchangeDsc
Node MSEEx1 {
LocalConfigurationManager
{
ConfigurationMode = 'ApplyAndMonitor'
RebootNodeIfNeeded = $true
ActionAfterReboot = 'ContinueConfiguration'
ConfigurationModeFrequencyMins = 1200
RefreshFrequencyMins = 1200
}
ExchMailboxDatabase DBSetup
{
Name = 'DB1'
Credential = $Creds
EdbFilePath = 'C:\Program Files\Microsoft\Exchange Server\V15\Mailbox\DB1\DB1.edb'
LogFolderPath = 'C:\Program Files\Microsoft\Exchange Server\V15\Mailbox\DB1'
Server = 'MSEEx1'
CircularLoggingEnabled = $true
DatabaseCopyCount = 2
PsDscRunAsCredential = $Creds
}
ExchMailboxDatabaseCopy DBCopySetup
{
Identity = 'DB1'
Credential = $Creds
MailboxServer = 'MSEDev'
ActivationPreference = 1
PsDscRunAsCredential = $Creds
}
}
}
$configData = @{
AllNodes = @(
@{
NodeName = $env:COMPUTERNAME
PSDscAllowPlainTextPassword = $true
PSDscAllowDomainUser = $true
}
)
}
Remove-Item -Path C:\DSC\* -ErrorAction SilentlyContinue
mkdir -Path C:\DSC -ErrorAction SilentlyContinue
mkdir -Path C:\ExchangeDatabases -ErrorAction SilentlyContinue
$cred = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
ExchangeMailboxConfig -ConfigurationData $configData -Creds $cred -OutputPath C:\DSC
Set-DscLocalConfigurationManager C:\DSC -Verbose
Start-DscConfiguration C:\DSC -Force -Wait -VerboseAlso the script to only test the connection and return a path object works as expected again: configuration ExchangeMailboxConfig {
param
(
[PSCredential]$Creds
)
Import-DscResource -Module ExchangeDsc
Node MSEEx1 {
LocalConfigurationManager
{
ConfigurationMode = 'ApplyAndMonitor'
RebootNodeIfNeeded = $true
ActionAfterReboot = 'ContinueConfiguration'
ConfigurationModeFrequencyMins = 1200
RefreshFrequencyMins = 1200
}
Script SetupExchangeConnect
{
GetScript = {
return @{ Result = '' }
}
TestScript = {
return $false
}
SetScript = {
function Invoke-DotSourcedScript
{
[CmdletBinding()]
[OutputType([System.Collections.Hashtable])]
param
(
[Parameter(Mandatory = $true)]
[System.String]
$ScriptPath,
[Parameter()]
[System.Collections.Hashtable]
$ScriptParams = @{ },
[Parameter()]
[System.String[]]
$SnapinsToRemove,
[Parameter()]
[System.String[]]
$CommandsToExecuteInScope,
[Parameter()]
[System.Collections.Hashtable]
$ParamsForCommandsToExecuteInScope
)
[System.Collections.Hashtable] $returnValues = @{ }
. $ScriptPath @ScriptParams
for ($i = 0; $i -lt $CommandsToExecuteInScope.Count; $i++)
{
[System.String] $commandToExecute = $CommandsToExecuteInScope[$i]
[System.Collections.Hashtable] $commandParams = @{ }
if ($ParamsForCommandsToExecuteInScope.ContainsKey($commandToExecute))
{
[System.Collections.Hashtable] $commandParams = $ParamsForCommandsToExecuteInScope[$commandToExecute]
}
$returnValue = . $commandToExecute @commandParams
if (!$returnValues.ContainsKey($commandToExecute))
{
$returnValues.Add($commandToExecute, $null)
}
$returnValues[$commandToExecute] = $returnValue
}
if ($SnapinsToRemove.Count -gt 0)
{
Remove-HelperSnapin -SnapinsToRemove $SnapinsToRemove -Verbose:$VerbosePreference
}
return $returnValues
}
$DSCExchangeModulePath = 'C:\Temp'
$Credential = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
$serverFQDN = 'MSEEx1.contoso.com'
$exbin = Join-Path -Path ((Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\Setup).MsiInstallPath) -ChildPath 'bin'
$remoteExchange = Join-Path -Path "$exbin" -ChildPath 'RemoteExchange.ps1'
$commandToExecuteAfterDotSourcing = @('_NewExchangeRunspace')
$commandParamsToExecuteAfterDotSourcing = @{
'_NewExchangeRunspace' = @{
fqdn = $serverFQDN
credential = $Credential
UseWIA = $false
AllowRedirection = $false
}
}
$returnValues = Invoke-DotSourcedScript `
-ScriptPath $remoteExchange `
-CommandsToExecuteInScope $commandToExecuteAfterDotSourcing `
-ParamsForCommandsToExecuteInScope $commandParamsToExecuteAfterDotSourcing `
-Verbose:$VerbosePreference
if ($null -ne $returnValues -and $returnValues.ContainsKey('_NewExchangeRunspace'))
{
$session = $returnValues['_NewExchangeRunspace']
}
if ($null -ne $session)
{
$session.Name = 'DSCExchangeSession'
}
New-Item -Path $DSCExchangeModulePath -Type Directory -Force | Out-Null
Export-PSSession -Session $Session -OutputModule $DSCExchangeModulePath -Force -AllowClobber | Out-Null
Import-Module -Name $DSCExchangeModulePath -Global -DisableNameChecking -Function *
#Wait-Debugger
$dbs = Get-MailboxDatabase
Write-Host '----------------------------------------------------------------------------------------------------------------'
'LogFolderPath:' | Write-Host
$dbs[0].LogFolderPath | Out-String | Write-Host
Write-Host '----------------------------------------------------------------------------------------------------------------'
}
PsDscRunAsCredential = $Creds
}
}
}
$configData = @{
AllNodes = @(
@{
NodeName = "$env:COMPUTERNAME"
PSDscAllowPlainTextPassword = $true
PSDscAllowDomainUser = $true
}
)
}
Remove-Item -Path C:\DSC\* -ErrorAction SilentlyContinue
mkdir -Path C:\DSC -ErrorAction SilentlyContinue
$cred = New-Object pscredential('contoso\install', ('Somepass1' | ConvertTo-SecureString -AsPlainText -Force))
ExchangeMailboxConfig -ConfigurationData $configData -Creds $cred -OutputPath C:\DSC
Set-DscLocalConfigurationManager C:\DSC -Verbose
Start-DscConfiguration C:\DSC -Force -Wait -Verbose |
I would guess it is the same principle as a SecureString cannot be decrypted using another account that the one that encrypted it. Since it works now my guess is that the account used to sign the serialized payload must also be used de deserialize the payload. But doesn’t answer your question though, but great that it works and we got closer to understand the reason. 😊 |
Why does the code run without problems in the context of the local system, the local administrator and any other administrator? Only when it is executed within the LCM does the serialization and deserialization not work. |
|
You mean it does not work when LCM runs the resource using SYSTEM as it does default if the |
|
Correct, and since the LCM is a black box for us, I doubt that we can find the cause. Since DSC configurations don't work without credentials anyway, the workaround with @gborus, how did your tests work out? |
|
Likewise, I've just finished testing with
Big thank you to @raandree and @johlju for getting to the bottom of this. |
Hi,
Is anyone aware of any recent changes (possibly November updates) that have started causing issues with this module? We've been using it for some time and I've recently noticed there are multiple resources where the test-targetresource function is continually returning as false. I've started looking further into the ExchExchangeCertificate resource which is one of them but the others listed below are also failing to test correctly on each run
My initial thoughts are that it might be something to do with how it's comparing arrays, but I haven't been able to determine anything conclusive yet. I can start digging further if needs be but I'm wondering if this is a known problem before I do so?
ExchReceiveConnector
ExchOutlookanywhere
ExchDatabaseAvailabilityGroupMember
ExchAutoMountPoint
ExchMailboxDatabase
ExchMailboxDatabaseCopy
The text was updated successfully, but these errors were encountered: