If there is desire I can create a simple plug-and-play template .wl file on GitHub that people can use to write their own structures.

@b3m2a1 Thanks, I will investigate it step by step

Let's refine it

1) Do you know why *Entry* functions aren't Hold*?

2) And why does it matter for your *construct* questions to be HoldFirst?

@b3m2a1 ^

5) I don't think this sentence is clear for wide audience: "mutability means it's harder to serialize the state"

6) Format should generally return an expression, not a box structure. So why not MakeBoxes upvalues?

may need to access data, what is the idiomatic way to do this

Just FYI, there are also System`Private`HoldNoEntryQ and System`Private`HoldSetNoEntry

+1

@LeonidShifrin nice to see you, somehow I got a feeling I will lure you.

@Kuba :)

Nice to see you too

I've been very busy lately for the new relational db stuff

I see, I was wondering if that is the case.

B.t.w., in my view and also some of my colleagues, atomic structures are at best a mixed blessing

@LeonidShifrin is there an alternative?

For example, in Databases project, we have a bunch of objects, and use it for only one, and the reason we do that is that that particular object should not be manually modified by the user, and we need to be able to check that

@Kuba Well, the question is why would one normally need to use it. The only good reason it is currently needed in some cases is that the constructor has the same head as the formed object, and one needs to differentiate

And the encapsulation arguments I don't buy really - either the object is used through the exposed API, and then it is not needed, or it is not, and then some crap may happen regardless of whether you make it noentry or not

I would rather have a dedicated head like Constructor[head, args]

and not use the noentry, except pretty special cases

Like the one I mentioned in Databases, for example

@LeonidShifrin how to make sure head[data] is valid ?

and that my api works with it and otherwise issues a message

I guess that is what I need

Depending on the case, you could either simply check for head[data] structure in the pattern (if it is fast to check), or you could set a Valid flag

One of the biggest issues with NoEntry is that it is really a poor man's shortcut for encapsulation, which breaks the correspondence between Full or InputForm and the ability to destructure the expression using rules. This is expressly against the rest of the language design.

This answer (particularly the "OOP considered harmful" section) is something I very much agree with.

That is not to say that there aren't cases when noentry stuff is useful. I would just not make decisions about this blindly.

@LeonidShifrin yes I think I don't care about entry. I just wasn't aware of Valid and condition/patter check scared me.

@Kuba FWIW, we mostly use condition / patterns at work. This is because we typically use a relatively small number of objects types, to structure our app on a rather high level. So there isn't that much of an overhead in checking these patterns, and OTOH we are not so much worried about malformed objects, because there aren't many ways to get those.

What we do extensively though is throwing exceptions in inner functions when something goes wrong. So we are likely to catch such bad cases early, if they ever occur.

@LeonidShifrin the problem I have is that I don't have strong OOP background and anything more sophisticated in MMa is hidden. So I don't know what is the way to go to implement something robustly.

@Kuba I think that the advice of @b3m2a1 to use SubValues or UpValues is good. We use both at work, although tend to use SubValues more. I prefer method names that are inert symbols scoped in some namespace, while my colleagues prefer string method names. We don't need inheritance all that often, mostly we just need the ADT (data and methods encapsulated in one place).

We tend to use immutable objects - in fact, currently there is no single place in our code base where mutable objects or state of any kind is used. But I admit that in some cases mutability might be required

@LeonidShifrin I was playing with a finite state machine implementation, if you are interested in the context. So mutability is an important point here as well.

Well, I would think that the finate state machine is well-suited for immutable structures.

If you are interested in how we set things up (which might not be ideal but it suits our current needs), you can read the source of Databases. You can search for DatabaseStore, RelationalDatabase, DBType, DBQueryBuildObject, DBQueryPrefixTrie for some example of objects we have, and their implementation.

right?

I need to understand it better but somehow mutable structure felt more natural for me.

Yes, but more commonly you would just chain

Have a look at how DBQueryBuilderObject is implemented

It is immutable and yet there are no problems chaining method calls

The key point is that every method that does something to the structure, rerturns a new modified one

So you can chain them

@LeonidShifrin isn't an overhead of recreating an 'object' important?

it is that vs just mutate a nested association

otoh chaining fits nicely here

You typically don't recreate a full object, just change some internals. And our objects are based on assocs, so this is cheap. Also, we mostly use objects for high-level project structuring, so we don't get that many of them

Here is a typical example of how method implementation can look using this way of coding:

oldQ_DBQueryBuilderObject @ wrapInQuery[selectedFieldsToExport_: None] := With[
    {q = oldQ @ DBResolveActualFields[True, False]},
    DBQueryBuilderObject @ new["Query"] @ addTable[
        q , selectedFieldsToExport
    ] @ set[
        "FieldPrefixTrie" -> q @ get["FieldPrefixTrie"]
    ] @ DBDistinct[
        (* Note: inherit DISTINCT property from inner query, by default *)
        q @ get @ "Distinct"
    ] @ set[q @ get[{"PrimaryKey", "SingleRow"}]] @ set["Schema" -> Inherited]
]

You can see a lot of chained methods

each one returns the next stage of the object

yet it is easy to read (IMO)

@LeonidShifrin I suppose it is much easier / comfortable to make such choices if you really understand how it works internally :)

Here is the set of core methods that we add:

addCoreMethods[typeSymbol_Symbol] :=
    Module[{},
        typeSymbol[o_, meta: $blank] @ getDataAssoc[] := o;

        (q:typeSymbol[_, meta: $blank]) @ set[field_String -> sub_ -> val_] :=
            Module[{assoc = q @ getDataAssoc[]},
                If[!KeyExistsQ[assoc, field],
                    DBRaise[
                        set,
                        "can_not_add_new_fields_to_" <> ToString[typeSymbol],
                        {q, field}
                    ]
                ];
                assoc[field][sub] = val;

and the type is typeSymbol[data_Association, meta_:None] basically

Then you can transform your structure by a chain of calls to set or transform or append etc methods, which can modify properties of sub-properties in the data assoc.

@LeonidShifrin very nice

I need to digest it of course but looks promissing

And that's really it. You don't need anything else.

Well, you need also a constructor

There is one more thing you need to enable the chaining to work

for our case, we use this: q_DBQueryBuilderObject[(fst: f_[___])[rest_]] := q[fst][rest]

This makes sure that the precedence of chain of calls using @ is correct

@LeonidShifrin why not new[] // method[] // method[]?

Also possible

Feels more natural, doesn't it?

new[] @ method[] reminds me of NetLink etc

And in fact, the way I just shown above, is my preferred one, but is disliked by my colleagues, who prefer something more conventional, like what you suggested

Right!, So you are telling me to be more idiomatic and then you do this :P

I think, it really boils down to what is most natural to you. The idiomatic part is to use immutable structures when you can. The rest is syntactic sugar, and is a matter of personal taste

@LeonidShifrin is there a good read about that principle? I always felt that mutating is superior.

@Kuba In my future book, which is currently in my head :) I could make an SE post though, listing the alternative approaches with examples.

Mutating has a number of issues in WL, from performance to GC-related

@LeonidShifrin I would gladly read it but I don't want to drain your time

@Kuba I will keep that in mind for the nearest window I have. Not sure when that will be, but I can try to make it soon.

@LeonidShifrin ok, thanks :)

@Kuba :)

@Kuba tried to include your points and make my answer more clear

@LeonidShifrin I tend to dislike this as it adds (to my mind) more complexity with no real benefit. This may also be because my I learned programming in python first and that is how python works, though. I'm also a big fan of unifying an entire API to a single Symbol for simplicity. I find programming in Mathematica to already require people to remember too many things, so this is a way to cut down on this.

@Kuba That's exactly why I like to write this way

It feels write to my mind that's use to think of stuff like obj.field.modify()