• panda_abyss@lemmy.ca
    link
    fedilink
    arrow-up
    4
    ·
    edit-2
    3 days ago

    So as someone who does a lot of stats in the private sector, and has known a lot of stats can employees…

    They are still woefully behind industry in practices and efficiency.

    It’s still incredibly difficult to just pull data from statscan, and they could be storing/distributing it 100x more efficiently with modern open formats like parquet – which would massively cut costs. The vector system they use is incredibly confusing.

    They do now have containerized notebookd and an on demand cluster. So that’s something we had 10 years ago.

    From the people I know there, they don’t feel they’re learning, they feel their skills are useless and atrophying, and leadership sounds like it’s filled by brown nosing morons. There’s also a lot of people who I would never have hired there – but same with big corps.

    It’s funny, I’d love access to their data, or to do work for them, but I could never work there.


    ETA: I should clarify, I don’t think the problem is entirely too many people, it’s shit leadership, overly burdensome rules, lack of tech innovation (admittedly, they are improving).

    I think one issue that really bites government workers is the idea that you spend your whole career in one place, from university to retirement. Frankly, that’s a horrible idea in the 21st century. You need to bring in people with expertise, you need diversity of thought, and you need people who have worn many hats in their lives.

    The field of analytics has changed a lot in 10-15 years, and Stat Can has not had a good pace.

    • corsicanguppy@lemmy.ca
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 days ago

      They do now have containerized notebookd and an on demand cluster. So that’s something we had 10 years ago.

      Containers violate iso27002. The pros know that chasing the sparkle is a bad idea.

      spend your whole career in one place, from university to retirement. Frankly, that’s a horrible idea in the 21st century. You need to bring in people with expertise

      No. You train your staff because you value them – you don’t ditch experience. You’ll discover the truth of this eventually.

      • panda_abyss@lemmy.ca
        link
        fedilink
        arrow-up
        1
        ·
        1 day ago

        Also, how do containers violate iso27002? Not familiar with it.

        And when I said containers, what I mean is the ability to spin up an environment on-demand with specified resources and a notebook environment.

      • panda_abyss@lemmy.ca
        link
        fedilink
        arrow-up
        1
        ·
        1 day ago

        I really want to agree with you.

        No. You train your staff because you value them – you don’t ditch experience. You’ll discover the truth of this eventually.

        I wish this was common sentiment. The modern company only values you on the way in or the way out. Otherwise you don’t have any leverage.

        I’ve always believed in the idea of hiring either experienced technical leaders you want to keep around and who will mentor, and junior staff who can be mentored into roles where they’ll be happy+productive for the long term.

        Around 2021 I stopped being a manager and companies stopped doing this.

        But this does come back to my point: Stat Can needs fresh blood to bring in new ideas. I don’t think they have those modern technical leaders the way they should, and they’re still stuck on legacy tools that make it impossible to pull in private sector veterans.