Collection literals

[nikitabobko]

This is an issue for discussion of collection literals proposal. The full text of the proposal is here.

Please, use this issue for the discussion on the substance of the proposal. For minor corrections to the text, please open comment directly in the PR #417.

[xvbcfj]

Finally!! 🎊🎊🎊

Really wish that tuples and maps weren't excluded though. And also addition of a slicing syntax would have been nice, although technically it is only somewhat related here.

Is the plan to release this as preview in 2.2.0?

[CLOVIS-AI]

First, as someone who occasionally teaches Kotlin to developers who are coming from other languages (most notably Java, JS and Swift), no one ever mentioned any difficulties learning the language because of the lack of collection literals. The main syntax difficulties I've seen was with the mandatory parenthesis after keywords (optional in Swift and Rust).

Clear intent. A special syntax for collection literals makes it clear that a new instance consisting of the supplied elements is created. For example, val x = listOf(10) is potentially confusing, because some readers might think that a new collection with the capacity of 10 is created. Compare it to val x = [10].

I don't think this is so black-and-white. Java uses int[] x = new int[10] to declare an array of size 10, and Kotlin doesn't have new and avoids writing the types, so [10] could very well be read by a Java developer as creating an array of size 10.

That's not to say I'm against collection literals (I'm not), but they are definitely not a major issue at the moment, and I think that overestimating their need will lead to rushing the implementation. Compared to how little the lack of collection literals impacts everyday code, adding them now will introduce a major change in how Kotlin code looks, and will create a big before/after rift. Newcomers will always have to learn about emptyList and listOf: even 10 years into the future, there will still be resources from before literals, and there will still be developers who still have the habit.

[CLOVIS-AI]

// before 1
   if (readlnOrNull() in listOf("y", "Y", "yes", "Yes", null)) {
       // ...
   }
// after 1
   if (readlnOrNull() in ["y", "Y", "yes", "Yes", null]) {
       // ...
   }

An important part of the Kotlin design is how the shortest code is (almost) always the best code. Is in [] the best code? What's [] here?

By making this code simpler we are emphasizing it to be correct to newcomers. Is this the kind of code we want Kotlin developers to write? AFAIK, this example is currently discouraged in code review, instead recommending to create a HashSet beforehand, or using a when statement, which will generate a switchtable:

when (readlnOrNull()) {
    "y", "Y", "yes", "Yes", null -> …
}

AFAIK, the when statement is optimal here, because it uses less memory and is faster than all other options. I think the explicit listOf is actually useful here: it emphasizes that a temporary list will be created, which leads users to search for a more optimal way to write this code (this is a fairly common question in the Kotlin Slack's #getting-started, I think I've answered it two or three times in the past year).

If we're going to have a shorter syntax here, then it should be the optimal way of writing this code. For example, the compiler could recognize that we're using in on a collection that contains only literals, and replace the expression by a switchtable.

My point isn't really about this specific example, but more about the general approach: we've lived well without collection literals so far, so if we're adding them now, how can they help us make the language actually easier, not just a few keystrokes shorter?

I think having in ["foo", "bar"] be a compiler intrinsic, and not 'just' a collection literal, provides a lot more value to this feature. First, because it makes the optimal solution for a common pattern simpler. Second, because this is code newcomers intuitively want to write already, and there is thus no learning curve.

I think we should take advantage of the fact that newcomers will want to use collection literals everywhere, to silently make these patterns optimal without actually using collections everywhere we can. I'm sure there are many more cases of the same pattern. And, worst case, you can always opt-out by using listOf.

[xvbcfj]

overestimating their need will lead to rushing the implementation

Collection Literals are being proposed by people in one form or another since over a decade ago and is one of the most requested feature. Not sure what you mean by "rushing" here. They are way overdue.

// before 1
if (readlnOrNull() in listOf("y", "Y", "yes", "Yes", null)) {
// ...
}
// after 1
if (readlnOrNull() in ["y", "Y", "yes", "Yes", null]) {
// ...
}

An important part of the Kotlin design is how the shortest code is (almost) always the best code. Is in [] the best code? What's [] here?

By making this code simpler we are emphasizing it to be correct to newcomers. Is this the kind of code we want Kotlin developers to write? AFAIK, this example is currently discouraged in code review...

This is common idiomatic code in several languages including Python, Rust etc. Although it is more common to use tuples instead of magic desugaring.

If we're going to have a shorter syntax here, then it should be the optimal way of writing this code. For example, the compiler could recognize that we're using in on a collection that contains only literals, and replace the expression by a switchtable.

See the desugaring link I shared above.

[kevincianfarini]

Hi, I'm chiming in to mention that I don't really think that Kotlin's lack of collection literals is a chronic problem that warrants a new syntax to solve. I unfortunately don't find any of the items enumerated in the Motivation section of the proposal compelling, whereas I think there's some amount of drawback to the new syntax that's added to support this.

I share some of the opinions from @CLOVIS-AI above, particularly:

[Collection literals] are definitely not a major issue at the moment, and I think that overestimating their need will lead to rushing the implementation.

I'm not sure if working on this proposal drains resources from other areas of language development, particularly things like robust pattern matching, but I'd love to see resources invested in areas that are major problems for the Kotlin language rather than collection literals 😄.

[rnett]

I think that some of the restrictions on the of operators are unintuitive and complicate the mental model when working with operators enough that I think it warrants not treating them as an operator. Especially:

Restriction 1. Extension operator fun of functions are forbidden. All operator fun of functions must be declared as member functions of the target type Companion object.

(which is particularly unfortunate since it prevents ever creating collection literals for external types)

Restriction 3. All of overloads must have non-nullable return type equal by ClassId to the type in which static scope the overload is declared in.

Restriction 4. All of overloads must have the same return type. If the return type is generic, the constraints on type parameters must coincide.

Restriction 7. All of overloads must have no extension/context parameters/receivers.

Restriction 8. operator fun of functions are not allowed to return nullable types.

When you put all of these together, they don't act like regular Kotlin functions. And I think the mental model of "functions (including operators, which are just functions you call differently) behave like this, except this particular function that behaves differently" is much harder to work with than "functions behave like this, and collection constructors behave like this".

Also, when you put all of those restrictions together, it ends up looking a lot like the restrictions on constructors. Was any consideration given to making them constructor-like? E.g. collection-constructor(vararg ...) in the type.

[rnett]

Different issue: when I want a particular type of collection that can't be inferred, it seems quite awkward to get it via a collection literal. Lets say I want to pass a TreeSet to a method that takes Set. IIUC I would need to define a TreeSet local variable for the collection literal and then pass that in. This is very awkward. If I can cast it inline ([a, b] as TreeSet) it's a little better, but still awkward. Was some syntax to specify the type you wanted considered? For example, TreeSet[a, b]. That syntax works particularly nicely if we treat the operator methods as constructors instead.

[MichaelSims]

I've wanted collection literals for a long time, so I'm happy to see this. But I am disappointed that maps literals are out of scope. I would guess that the use of collection and map literals in most of my code is probably 50/50 so this definitely seems like a half solution to me.

[rnett]

I can see why they would want to start with just list-like collections, but I agree, I hope we get maps eventually too.

[CLOVIS-AI]

My understanding through talking with other people in the ecosystem and reading prior issues was that collection literals were not being implemented yet because it was complex to decide how they would work with custom types, and because the team was researching an alternative to varargs as an initialization mechanism. Since this design is based on varargs, have those concerns been eliminated somehow?

The Performance section mentions that varargs are faster for array-backed collections, but that's just ArrayList, isn't it? All other data structures we use daily (HashSet, LinkedHashSet, HashMap, LinkedHashMap and all custom types) are most likely not array-backed (or we would use ArrayList). We're talking about introducing new syntax here, so rules can be bent a bit. Is there really no design that can improve these use-cases?

[CLOVIS-AI]

What will be the behavior with custom types delegating to existing collections?

value class MyCustomList(private val data: List<String>) : List<String> by data {
    companion object {
        operator fun of(vararg items: String) = List.of(*items)
    }
}

The spread operator enforces an array copy here, right? Doesn't that negate the claims in the Performance section that a single array will be created? value class so far could be used as a free (or near-free) passthrough in hot paths to make code readable without performance impacts.

[CLOVIS-AI]

I want to thank everyone involved on this KEEP for making so thorough. I can't help but think, however, that this is a lot of complexity for the single purpose of removing 6 characters in listOf(10). The provided conversion examples are not that much better and in many cases require explicit type declaration, which is overall discouraged in Kotlin. It also seems that such a complex resolution algorithm will close many doors in the future by making resolution more complex than it currently is, no?

Especially since this is all varargs all the way down anyway, has the team considered adding the get operator hack to the standard library?

interface List<T> : Collection<T> {
    …
    companion object {
        operator fun <T> get(vararg items: T): List<T>
    }
}

It seems to me that this would provide all the same benefits, with the only downside that [5] must be written List[5]. Everything else brought by this KEEP is already available that way, without any of the ambiguities and without requiring any language change.

I understand that not having to write the type is part of the Kotlin spirit, but this KEEP has a lot of limitations and downsides to achieve something we can almost do today, and this KEEP includes a mention that it will forbid this syntax on types for which collection literals are made available, which means we won't be able to use this trick even when it leads to shorter code.

[jingibus]

I love the idea of including list literals, but the whole question of what to do about custom types, mutable collections, etc... it dramatically complicates the feature and dramatically complicates the experience of reading and understanding the call site for very little win.

Limiting literals to immutable collections simplifies things for everyone — beginner programmer, staff engineer reading PRs for footguns, spec writer, and Kotlin compiler implementer. The literal is an immutable list literal, full stop. It solves the 95% case, data literals in code. And if you want the 5% case, no problem: you can use extension functions to convert the literal into the desired type, or just use the old constructor functions.

It won't win a code golfing contest, but character count isn't THAT important. Certainly not more important than the mental overhead imposed by contextual semantics.

[eav-eav-eav]

The implied restrictions and complicated resolution rules make me even more yearned for #348 (Type guided APIs List.of, List.empty, MutableList.of, ArrayList.of, and being able to of on external types) or just always requiring the type (List [], Set [], but no bare []) for a much simplier mental model.