-
Notifications
You must be signed in to change notification settings - Fork 0
Programming_and_build
The GDD framework describes Awarness, Working, Practitioner, and Expert levels for Programming and build (data science).
- Awareness: show a basic understanding of software development principles and can write simple scripts under supervision.
- Working: write and test scripts, create basic models in one or more languages and collaborate on shared codebases, using a variety of methodologies
- Practitioner: write moderate to complex programs, scripts and show a good understanding of testing methodologies and how to deploy code.
- Expert: write complex programs and scripts, seek to make code open source where appropriate, supervise junior analysts and set coding standards for your team understand software architecture and how to write efficient, optimised code and be able to perform user testing on products prior to launch.
The table below shows which skill level each grade should be working at within three to six months of starting a new role.
| Grade | Trainee Data Scientist (EO) | Associate Data Scientist (HEO) | Data Scientist (SEO) | Senior Data Scientist (G7) | Lead Data Scientist (G6) |
|---|---|---|---|---|---|
| Skill level | Awareness | Working | Practitioner | Expert | Expert |
If you are looking for some beginners tutorials on using Python please have a look at Data analysis and synthesis awareness.
If you are looking for training that leads to certification on GCP you may be able to take part in the Google Get Certified Program.
For some general git training across a range of topics on Git and GitHub please see GitHub Skills
If you are going to want to start writing Python you are going to need something to write it on this is where integrated development environments (IDEs) come in. You will probably be using one or more of these:
- Visual Studio Code (or VS Code for short). You can install VS Code on your ONS laptop by logging a service desk request for the software and then downloading and installing via the software center. Here is some basic introductory training on how to use VS Code to write Python.
- Jupyter Notebooks (or similar notebook type interfaces). These must be installed by requesting Python on the service desk and then follow the ASAP wiki. Learning Hub: Python - Jupyter Notebooks Overview.
- Spyder. As with Jupyter Notebooks you install Spyder using the service desk and the ASAP wiki. YouTube: basics of Spyder
An overview of the philosophy of Python programming can be found in PEP 20: The Zen of Python. See PEP 8 for a more detailed style guide.
The ability to write object oriented code is a key feature of Python. Here is a short video comparing functional programming with object oriented programming, YouTube: FP vs OOP.
Learning Hub: Command line basics you can also use this list if you switch between using Windows and Linux commands often and need to translate.
Learning Hub: Introduction to Continuous Integration
At the ONS we use Git, along with GitHub and GitLab for version control. Version control is a part of writing reproducible analytical pipelines (RAPs).
- Learning Hub: Packaging and Documentation
- Article: What is Git? Our beginner’s guide to version control
- Learning Hub: Introduction to RAP
- Learning Hub: Loops and Functions
Unit testing is also an important part of pipeline building with Python. You can find more about this on the Testing page.
There is a great series of YouTube videos where you can get a quick overview of GCP services called Cloud Bytes. We include some here:
- YouTube: Cloud Storage in a minute
- YouTube: Cloud Functions in a minute You can also learn more about Cloud Tasks and Cloud Scheduler in this video.
- YouTube: Pub/Sub in a minute There will be many data engineers in the ONS who have no need for Pub/Sub because they wont be streaming data.
- YouTube: BigQuery in a minute
- YouTube: BigQuery ML in a minute
- YouTube: Cloud IAM in a minute
- YouTube: Cloud Logging in a minute
- YouTube: Cloud Monitoring in a minute
- YouTube: Dataproc in a minute
- YouTube: Dataplex in a minute
At working level you may already be familiar with some Python design patterns but for all the ones you are not familiar with you can use the link.
Learning Hub: DataFrames, Manipulation and Cleaning
Learning Hub: Introduction to Object Orientated Programming in Python
- Learning Hub: Introduction to Git
- Presentation: Make Git Happen
- Game: Oh My Git for use on an off-network laptop only.
- Cheat Sheet: Git
- Learning Hub: Packaging Code with Python
Cloud Skills Boost (10 credits): Dev Apps with Cloud Functions