ConnId 2.0.0

Owned by Francesco Chicchiriccò
Last updated: Feb 03, 2020 by Radovan Semancik
5 min read

This page contains a notes regarding our current thinking about ConnId 2.

ConnId 2 should be a “next generation” of ConnId platform. It should support operations and use-cases that are not possible with ConnId 1 - and it is not feasible to implement them and keep connector compatibility at the same time. Therefore ConnId 2 can break the compatibility (in a reasonable way, see below).

Evolution

ConnId2 should be an evolution of ConnId1. We are not all the rewriting the code.

Rewrite would be attractive. We can get rid of CDDL, we can clean up a lot of things, modernize from the ground up. But that would be a huge task. We do not have human resources (and funding) to do that.

Therefore we rather go for evolution. Rough plan:

  1. Analyse and design. Prepare the list of all incompatible changes that we need to do in ConnId2.

  2. Modify the interfaces by applying all the incompatible changes. Modify the implementation as well (if possible). Do not add new functionality yet. Just change the interfaces in such a way that any changes that follows can be made in a compatible manner.

  3. Stabilize the functionality.

  4. Release ConnId 2.0.0.

  5. Add more functionality. In small steps. With compatible interface changes.

  6. Goto step 5.

Schema

There are two issues here.

Complex attributes

We want to support complex attributes. E.g. attribute foo contains a map with keys bar1 and bar2, string values, int values and so on.

We have two options:

  • Adopt some kind of schema language (JSON schema, XSD, …)

  • Extend current schema capabilities of ConnId (e.g. AttributeInfo class)

Use of schema language is quite attractive. But there are potential obstacles. Which language we would use? Can we implement all the features of that language? Schema languages are usually tightly bound to representation format, e.g. JSON schema is bound to JSON. But our data are not JSON, they are Java primitive types spiced with collections. Will JSON schema fit? What features of JSON schema we won’t be able to support? If we can support only a fraction of JSON schema capabilities, we might end up looking like a mouse in a elephant’s skin.

It looks like extending current ConnId schema seems to be much easier. This will naturally limit the capabilities to the set that we need. This will also mean that porting of ConnId1 connectors should be easy. In fact, pretty much the same schema code could be used and only some minor adjustments should be needed.

Therefore the decision for now is to extend native ConnId schema. But that decision may change if we run into unforeseen difficulties.

Attribute values should be easy to do. We will just use Map/List and primitive data types.

Identifiers

We want cleaner handling of identifiers, especially better handling of __NAME__ and __UID__ atttibutes.

Firstly __NAME__ and __UID__ usually stand for some native attributes, such as DN and entryUUID. Masking those as __NAME__ and __UID__ makes troubleshooting difficult. We want to use native names instead.

Secondly, there are resources that only have one of the (e.g. only have mutable username). There are resources that have both, but __NAME__ is not unique. And so on.

Thirdly, there are resources that will benefit if both identifiers are passed to operations. Such as AD resource in case of AD forests. In that case GUID (__UID__) is primary identifier. But GUID is not reveal the server where an object is stored. Passing both GUID and DN can make operations much more efficient.

The decision for ConnId2 is to make the use of __NAME__ and __UID__ optional. They will still be there, they may even be present in the schema by default, but they will no longer be required.

The schema will have new capability to define identifiers. E.g. The AttributeInfo class for the dn attribute may specify that this is a secondary identifier.

Operations should be modified to allow passign more than one identifier. E.g. the Uid parameter in the methods should be replaced with something like Identification that will be a container for identifiers. Or there may be additional parameters for operation. Specific implementation is still TBD.

Remote Connector Servers

Connector servers are currently quite under-maintained. What we will do about that?

  • Keep Java connector server. This one is quite useful. But try to modernize it. Try to improve logging, error handling, packaging, etc.

  • Drop .NET connector server and all other .NET parts. We do not really need them and we do not have the manpower to maintain them. Deprecate .NET support in ConnId1, remove it completely in ConnId2.

API/SPI Operations

There is a need for some updates:

  • Create GetApiOp and GetOp. Hence split search and get operations.

  • Introduce CountApiOp and CountOp. Useful for GUI.

  • We have too many update operations. Let’s reduce that to just one set. That might be updateDelta operation.

Result Handlers

Nobody loves result handlers. We do not need them.

Deprecate them in ConnId1, remove them in ConnId2.

Asynchronous Operations

This is a difficult problem. So far we have the questions only:

  • How to support operations that do not return immediately?

    • E.g. operations that implement "manual provisioning"?

  • Should it be integral part of operations? e.g. operation that started as sync can end up as async?

  • Should this be based on polling? Donating thread to connector? Shared queue?

  • What about REST service endpoints and message queues?

We need to get back to drawing board and think about it. Let’s discuss that later (approx. summer 2020?)

Misc

Misc improvements:

  • Clarify definition of runAsUser, maybe rework the parameters to properly use identifiers

  • Improve the documentation

Testing

How the testing framework really works? We have some idea, but nobody has a complete knowledge. The testing should probably be improved as we plan to do more changes now.

How to include connector server in the tests?

This has to be discussed later.

Low Priority (out of scope)

Those things are out of scope. We will not handle those in our initial attempts to create ConnId2. Of course, them may be added later during ConnId2 lifetime.

  • Capabilities

  • Versioning

  • Error handling

  • Synchronization improvements

  • Service accounts

  • Transactions

  • Entitlements

Compatibility and Migration

ConnId2 will not be compatible with ConnId1. We are deliberately dropping compatibility to avoid accumulating more technical debt.

However, we still need two things:

  1. Reasonable way to port ConnId1 connectors to ConnId2 framework. The porting must not require a complete rewrite of the connector. The porting should be a matter of a couple of text replace operations, some minor adjustments and so on. Really old connectors may need some of their methods reworked. But overall, we want to keep porting overhead quite small, mostly a matter of few mandays or work.

  2. We absolutely need a way how to run ConnId1 and ConnId2 connectors together. ConnId1 connectors will be there for a very long time. Even though ConnId1 will not evolve any more, we should be able to run those legacy connectors.
    There a simple and elegant solution: change the package name of ConnId2 framework. We can use connid.net. Therefore the ConnId1 and ConnId2 platforms should be independent from Java runtime point of view.

Plan

There is no specific plan yet. There is no commitment yet. We are just exploring possibilities. We are making sure that we are aiming for the same goal, that we can agree on the approach and that our development efforts will converge.

So far, nobody is making any specific commitment about dates/resources and nobody is expecting any commitment.

Origin

This text originated from on-line discussion (call) in February 2020.

See connid-dev mailing list archives.