I'm in similar situation wrt what I use python for, and suffered from the same apathy until quite recently. Man, it was really easy to switch, and completely worth it just for the little things (like UTF-8 as default encoding for strings). It's really painless, but if you want to ease into it, I recommend using __future__ in your py2 stuff for now just to get you into a py3 state of mind: https://docs.python.org/2/library/__future__.html

UTF-8 isn't CPython's default encoding for strings. The internal encodings are ASCII, UTF-16, or UTF-32, depending on the widest character in the string.

I would have made UTF-8 the internal representation, and generated an array of subscript indices only when someone random-accessed a string. If the string is being processed sequentially, as with

    for c in s :
        ...

you don't need an index array. You don't need them for list comprehensions or regular expressions. One could even have opaque string indices, returned by search methods, which don't require an index array unless forcibly converted to an integer. Some special cases, such as "s[-1]", don't really need an index array either.

I agree. Go does something similar (using UTF-8). The Go creators _invented_ UTF-8, so I trust them. :)

Go uses UTF-8 internally, but you index a Go string by bytes, not glyphs or graphemes. The "index" function on strings in Go returns an integer, not an opaque index object or a slice. You can create a slice of a UTF-8 string which is misaligned and not valid UTF-8.

Indexing a string in Python 3 returns glyphs (which are strings), not bytes. Not graphemes; if you index through a Python 3 string with emoji that have skin color modifications, you'll get the emoji glyph and the skin color modifier as separate items.

Python 3 also has a type "bytes", which can't quite decide whether it's a string type or an array of integers. Print a "bytes" type, and it's printed as a string, with a 'b" in front of it, not as an array of integers. But an element of a "bytes" array is an int, not a bytes type. This is for backwards compatibility; it's roughly compatible with legacy "str" from Python 2.

Rust struggles with this. Rust tries to prevent you from getting a invalid UTF-8 string. The solution used involves re-checking for UTF-8 validity a lot, or bypassing it with unsafe code. This gets messy.

> The solution used involves re-checking for UTF-8 validity a lot,

You only need to check once, at the boundary of when you're converting from something that may or may not be UTF-8 to a UTF-8 string.

I was the same, and switched to py3 last year because i felt like it didn't really matter, so might as well get off of python2.

> I use python for my personal research, so my particularly situation affords me this laziness.

I am pretty much in the same situation and in hindsight I kinda wish I upgraded a little sooner. Fixing print brackets was the biggest (not that big) task, and from there on it was all python3 sugar for me :)

What system do you use? 2.x is installed by default on MacOS, but not Ubuntu 16.04.