New Module to Implement MS-NEGOEX Protocol

[ThatTotallyRealMyth]

Hi there!!, as discussed from the previous PR #2170 I have got this partial implementation of the MS-NEGOEX protocol up! Its no where near ready and I have only implemented the relevant structures, some helper functions for the protocol(such as checking the message headers or a function to create a message header) as well as some errors and a base class to act as a state machine to control the protocol flow.

I noticed in the spengo.py implementation, there was no "state" machine and so I wanted to ask before continuing with development what would the module/protocol need to look like?

An additional question how to implement the sub protocols that are to be negotiated by negoex. Officially only PKU2U needs NEGOEX and cloudAP uses it as well. Since neither are present in impacket yet, is there a certain design choice that needs to be taken to accommodate future additions?

Finally, NEGOEX has an additional network exchange that can be side stepped if we, as the initiator, send something the spec calls as an optimistic token that contains information on the protocol we want to negotiate. I haven't found an answer or a location in which I could understand how to generate this for PKU2U(which I guess is the only protocol someone could implement now that would use it).

I would appreciate any feedback on the above so I know how to carry forward with implementation :3

[anadrianmanrique]

Hello. This seems to be a big feature, so I would like to have it broken down into more than a couple of milestones/PRs:
PR 1: NEGOEX Codec + Tests
PR 2: NEGOEX State Machine + Tests
PR 3: SPNEGO Integration
PR 4: PKU2U Parser / Auth Scheme Module

If you are ok with the plan, we can consider this one as PR 1 and focus on:

• Fix module import/compile errors.
• Add imports/helpers:
  • os
  • _asBytes()
  • UUID normalization helper for uuid.UUID, bytes, and string GUIDs.
• Define known constants:
  • NEGOEX_OID
  • AUTH_SCHEME_PKU2U = uuid.UUID('235f69ad-73fb-4dbc-8203-0629e739339b')
• Complete builders:
  • createNegoMessage()
  • createExchangeMessage()
  • optionally createVerifyMessage()
  • optionally createAlertMessage()
• Make parsing strict:
  • reject invalid signatures
  • reject invalid cbHeaderLength
  • reject cbMessageLength < cbHeaderLength
  • reject truncated messages
  • reject offset/length pairs outside message bounds
  • reject unsupported protocol version
  • reject invalid message type for each structure
• Expose safe accessors:
  • ParsedMessage.message_type
  • ParsedMessage.raw_data
  • NegoMessage.getAuthSchemeList()
  • NegoMessage.getExtensionList()
  • ExchangeMessage.getAuthScheme()
  • ExchangeMessage.getExchangeData()
  • VerifyMessage.getChecksumValue()
  • AlertMessage.getAlertList()
• Remove or keep minimal NegoExContext.
  • If kept, it should not imply full negotiation support. Better to defer it to PR 2.

Test

• targeted module tests/SMB_RPC/test_negoex.py
• createNegoMessage() round trip:
  • initiator nego
  • acceptor nego
  • zero extensions
  • one or more auth schemes
  • includes PKU2U GUID
• createExchangeMessage() round trip:
  • INITIATOR_META_DATA
  • ACCEPTOR_META_DATA
  • CHALLENGE
  • AP_REQUEST
  • empty payload
  • non-empty payload
• parseNegoExToken():
  • single message
  • multiple concatenated messages
  • preserves raw bytes and offsets
  • unknown message type is returned as raw/unknown if that is intended behavior
• Error tests:
  • truncated header
  • invalid signature
  • invalid cbHeaderLength
  • cbMessageLength shorter than header
  • cbMessageLength longer than available data
  • bad auth scheme vector offset/length
  • bad extension vector offset/length
  • bad extension value offset/length
  • bad exchange offset/length
  • bad checksum offset/length
  • bad alert offset/length
  • unsupported protocol version
  • wrong message type for each structure
• Serialization stability:
  • assert fixed header sizes:
    • message header 40
    • nego header 96
    • exchange header 64
    • verify header 80
    • alert header 68
  • assert GUIDs are serialized little-endian where required.

We can discuss in detail the following PRs later. The idea is to have a solid foundation for NEGOEX/PKU2U functionality starting from this PR, to be used later in some interesting usecases.

Let me know your thoughts

thanks

[ThatTotallyRealMyth]

Hey @anadrianmanrique, Thanks for taking the time to lay it out all out that way and for the plan/feedback. I think the approach you laid out is solid as it makes it easier to build towards it bit by bit. Im happy to use this as PR 1 and start ticking off elements from the list for PR 1

Can I clarify for your point on "Remove or keep minimal NegoExContext.", if we were to remove it, then how would we be able to control/drive the state machine during PR 2 or would removing it just mean we remove for now and then re-add later on