-Proxy seems to be ignored on Get-MgPolicyClaimMappingPolicy

[drdamour]

Thanks for reporting the bug. Please ensure you've gone through the following checklist before opening an issue:

• Make sure you can reproduce this issue using the latest released version of Microsoft.Graph or Microsoft.Graph.Beta.
• Please search the existing issues to see if there has been a similar issue filed.
• For issues related to authentication and service errors, please refer to our troubleshooting guide. For service issues, please open a question at https://developer.microsoft.com/graph/support.

Describe the bug
Have fiddler listening on 8888 locally, but -Proxy isn't doing anything different

To Reproduce
Steps to reproduce the behavior:

1. Get-MgPolicyClaimMappingPolicy -Proxy "http://127.0.0.1:8888"
2. Get-MgPolicyClaimMappingPolicy -Proxy "http://localhost:8888"

Expected behavior
Expect to see the http request/response in fiddler

Debug Output

C:\src> Get-MgPolicyClaimMappingPolicy  -Proxy "http://127.0.0.1:8888" -Debug
DEBUG: [CmdletBeginProcessing]: - Get-MgPolicyClaimMappingPolicy begin processing with parameterSet 'List'.
DEBUG: [Authentication]: - AuthType: 'Delegated', TokenCredentialType: 'InteractiveBrowser', ContextScope: 'CurrentUser', AppName: 'Microsoft Graph Command Line Tools'.
DEBUG: [Authentication]: - Scopes: [Application.Read.All, Directory.AccessAsUser.All, Domain.ReadWrite.All, Group.Read.All, openid, Policy.Read.All, profile, RoleManagement.Read.Directory, User.Read, User.Read.All, UserAuthenticationMethod.ReadWrite.All, email].
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://graph.microsoft.com/v1.0/policies/claimsMappingPolicies

Headers:
FeatureFlag                   : 00000043
Cache-Control                 : no-store, no-cache
User-Agent                    : Mozilla/5.0,(Windows NT 10.0; Microsoft Windows 10.0.19045; en-US),PowerShell/7.3.6
Accept-Encoding               : gzip
SdkVersion                    : graph-powershell/2.3.0
client-request-id             : 3af0e37d-7af2-4370-b13f-804d6a192e71

Body:



DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Transfer-Encoding             : chunked
Vary                          : Accept-Encoding
Strict-Transport-Security     : max-age=31536000
request-id                    : 71d6d935-f50f-4a94-ae50-5d283cf2ce01
client-request-id             : 3af0e37d-7af2-4370-b13f-804d6a192e71
x-ms-ags-diagnostic           : {"ServerInfo":{"DataCenter":"West Central US","Slice":"E","Ring":"1","ScaleUnit":"001","RoleInstance":"CY4PEPF00005443"}}
x-ms-resource-unit            : 1
OData-Version                 : 4.0
Date                          : Tue, 19 Sep 2023 23:27:13 GMT

Body:
{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#policies/claimsMappingPolicies",
  "value": [
    {
      "id": "6e1fa6a9-a7af-4b6c-b0f7-",
      "deletedDateTime": null,
      "definition": [
        "{\"ClaimsMappingPolicy\":{\"Version\":1,\"IncludeBasicClaimSet\":true,\"ClaimsSchema\":[{\"Source\":\"user\",\"ID\":\"JobTitle\",\"JwtClaimType\":\"JobTitle\"}]}}"
      ],
      "displayName": "Add JobTitle to claims",
      "isOrganizationDefault": false
    },
    {
      "id": "b6a570f5-c755-4b49-9c25-",
      "deletedDateTime": null,
      "definition": [
        "{\"ClaimsMappingPolicy\":{\"ClaimsSchema\":[],\"ClaimsTransformations\":[],\"GroupFilter\":{\"MatchOn\":\"samAccountName\",\"Type\":\"prefix\",\"Value\":\"App \"},\"Version\":1}}"
      ],
      "displayName": "Policy",
      "isOrganizationDefault": false
    }
  ]
}

Module Version

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.3.0                 Microsoft.Graph.Authentication      {Add-MgEnvironment, Connect-MgGraph, Disconnect-MgGraph,…
Script     2.3.0                 Microsoft.Graph.Identity.SignIns    {Confirm-MgRiskyServicePrincipalCompromised, Confirm-MgR…

Environment Data

Name                           Value
----                           -----
PSVersion                      7.3.6
PSEdition                      Core
GitCommitId                    7.3.6
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Screenshots
N/A

Additional context
tried common proxy configs like $Env:HTTPS_PROXY and [System.Net.Http.HttpClient]::DefaultProxy = New-Object System.Net.WebProxy('http://localhost:8888') as well. Also tried with -ProxyUseDefaultCredentials switch

[SeniorConsulting]

Hi drdamour,

I don't see a -Proxy parameter for this cmdlet:
https://learn.microsoft.com/en-us/powershell/module/microsoft.graph.identity.signins/get-mgpolicyclaimmappingpolicy?view=graph-powershell-1.0

I don't recall seeing it for any of the Graph SDK cmdlets I've encountered.

As for how to get the Graph SDK working with a proxy, the .NET proxy call you're using looks similar (but not the same) as the one I'm using:

[system.net.webrequest]::defaultwebproxy = new-object system.net.webproxy('http://localhost:8888')
[system.net.webrequest]::defaultwebproxy.credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
[system.net.webrequest]::defaultwebproxy.BypassProxyOnLocal = $true

That being said, I typically run PowerShell 5.1 (as opposed to 7.3, which you're using), so the above may not actually be a fix.

Perhaps you might be able to use winhttp set proxy?

Is this problem specific to the Graph SDK? From what I'm reading, you're trying to trace the traffic yourself so that you can see the calls being made yourself.
For example, if you run
Invoke-WebRequest ipinfo.io

Can you see the packets going out via Fiddler? If not, this seems less of a Graph SDK problem, and more of a request on how to use Fiddler with PowerShell.

Not sure whether it's the right tool, but have you considered something like WireShark, or netsh trace start?

[drdamour]

@SeniorConsulting it's output in the help as an available switch

[system.net.webrequest]::defaultwebproxy is for .net and not .net core generally, but i did try that as well and it did not work either.

also tried winhttp set proxy and that didn't work either.

yes i i see traffic with Invoke-WebRequest so it's not a problem with fiddler, this powershell seems to be ignoring those common proxy configurations. it's definitely a problem constrained to the graph powershell commands.

[drdamour]

there's def special behaviour in this regarding ps core vs win ps

msgraph-sdk-powershell/src/Authentication/Authentication/Helpers/HttpHelpers.cs

Line 68 in 38f2077

? GraphClientFactory.Create(delegatingHandlers)

in pscore it just passes through but in win ps it forces HttpClientHandler. NOTE: doesn't seem like proxy is passed in either case...that's weird....

tracing pscore behaviour through entry is to

https://github.com/microsoftgraph/msgraph-sdk-dotnet-core/blob/1c0eb8d0a63216a340ec2cf494a2f4d08cc1eb87/src/Microsoft.Graph.Core/Requests/GraphClientFactory.cs#L55 proxy defaults to null (so that would explain why -Proxy doesn't seem to be doing anything...maybe it's support is on a roadmap.

that ends up going through a bunch of platofrm specific if blocks and looks like it selects SocketsHttpHandler here https://github.com/microsoftgraph/msgraph-sdk-dotnet-core/blob/1c0eb8d0a63216a340ec2cf494a2f4d08cc1eb87/src/Microsoft.Graph.Core/Requests/GraphClientFactory.cs#L231 which explains why none of the global proxy configs work.

Also looks like in win ps it'll use WinHttpHandler...which should respect winhttp set proxy

Proxy is mentioned in a LOT of docs as being supported throughout this repo cf https://github.com/search?q=repo%3Amicrosoftgraph%2Fmsgraph-sdk-powershell%20proxy&type=code
and eg

msgraph-sdk-powershell/src/Identity.SignIns/v1.0/custom/README.md

Line 26 in 38f2077

- Proxy

but doesn't seem to actually work...

[SeniorConsulting]

Right you are. It seems like you've delved far further into this one than I have. I'll leave it to the actual devs to respond :) Thanks for the detailed notes.

[peombwa]

Thanks for bringing this to our attention and thank you SeniorConsulting for the insights.

The recommended way of using the commands behind a proxy is via global proxy settings, which is honored by the underlaying HttpClient as described at https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient#global-default-proxy.

1. For PowerShell 7.x

$proxy='http://localhost:8888'
$env:HTTP_PROXY=$proxy
$env:HTTPS_PROXY=$proxy

Connect-MgGraph
Get-MgPolicyClaimMappingPolicy

2. For PowerShell 5.1

[system.net.webrequest]::defaultwebproxy = new-object system.net.webproxy('http://localhost:8888')
[system.net.webrequest]::defaultwebproxy.credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
[system.net.webrequest]::defaultwebproxy.BypassProxyOnLocal = $true
# or
# Use the system proxy settings (internet options).

Connect-MgGraph
Get-MgPolicyClaimMappingPolicy

Both options work as expected in my test environment. See https://woshub.com/using-powershell-behind-a-proxy/ for more details.

The -Proxy is a legacy parameter that's wrongly added by the code generator and should be ignored for now.

[drdamour]

@peombwa as stated in original issue i tried the suggested approach for powershell core 7.x, but it did not work as the code doesnt seem to use httpclient. did u attempt this suggestion and see success?

[peombwa]

@drdamour, all the commands use HttpClient. The HttpClient will fall back to the global proxy settings when proxy is set to null.

Yes, I tested both approaches and they work as expected:

1. Configure the proxy in a new PowerShell session,
2. Run the command with the Proxy Server Off, it should fail with couldn't establish connection:
   
3. Run the command with the Proxy Server On. You should see the request go through your proxy as show:
   

Please verify that your proxy server is configured correctly (filter rules etc.).

[drdamour]

@peombwa weird...i'm not having success

with proxy (fiddler) shut down request succeeds ( expected it wouldn't), with it on no capture request succeeds

do you have a log level enabled that i don't? those dark blueish stuff.

u sure it uses HttpClient, it sure looks like it used SocketsHttpHandler around here https://github.com/microsoftgraph/msgraph-sdk-dotnet-core/blob/1c0eb8d0a63216a340ec2cf494a2f4d08cc1eb87/src/Microsoft.Graph.Core/Requests/GraphClientFactory.cs#L231

[peombwa]

I'm also not sure why it isn't working for you. Are you sure the provided port isn't forwarding requests to the internet? Please verify that your machine's proxy is configured correctly.

I'm using https://mitmproxy.org to test the proxy. It shows the proxy working. Other customers have also had success using the proxy via the global settings.

SocketsHttpHandler is just the final handler of the HttpClient (Default for .NET Core 2.1+) - https://learn.microsoft.com/dotnet/api/system.net.http.socketshttphandler.

[drdamour]

Yes im sure many other powershell commands are appearing in fiddler proxy, but mg cmdlets seem to be a special case.

will try on a diff machine and have peers try.

what do i mean exactly by “machines proxy is configured correctly “?

also u didnt mention about debug level i think

[drdamour]

Also im confused if this is leveraging httpclient in the way u describe surely

[System.Net.Http.HttpClient]::DefaultProxy = New-Object System.Net.WebProxy('http://localhost:8888')

should work…but it also does not on my system.