Data Engineer Python and SQL ,numPy at N Consulting Ltd

Job Overview:

We are looking for an individual who has at least a basic to intermediate understanding of SQL, Python, Pandas, and NumPy. It would be a plus to have knowledge of advanced-level concepts, but it's not a strict requirement. 

Key Responsibilities:

1. Extract, transform, and analyze, manipulate, restructure data using SQL and Python
2. Clean, filter, transform, and reshape data using Pandas
3. Perform numerical computations and vectorized operations using NumPy
4. Create scripts to automate data loading and preprocessing tasks
5. Collaborate with team members to understand data needs and deliver insights
6. Prepare and present basic analytical reports or summaries

__________________________________________________________________________________

Core Skills – Must Have:

=>SQL (Basic to Intermediate): SQL Skills

1. Core SQL Clauses — Must Know 
   Proficiency in using:
   SELECT, WHERE, GROUP BY, ORDER BY, LIMIT/OFFSET, TOP, HAVING
2. Different types of Operators.
3. Joins — Very Important
   Must have hands-on knowledge of:
   INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, should understand how improper join conditions can lead to duplicate data.
4. Functions — Must Know 
   Built-in functions: Aggregate, Analytical, String, Date functions
   User Defined Functions (UDFs)
5. Filtering Techniques — Must Know 
   Use of:
   BETWEEN, IN, LIKE, IS NULL
6. Subqueries & Common Table Expressions (CTEs) — Must Know
7. Window Functions — Preferred (Good to Have), should know how it can be used in analytical and aggregate functions. (Good to have)
8. Basic Schema Design — Must Know
   Understanding of: Tables and relationships, Primary and foreign keys, Data types

=>Python (Basic to Intermediate — Very Important):

1. Data types: lists, dictionaries, tuples, sets
2. Loops, list comprehensions, and conditional statements
3. Functions, lambda expressions, and exception handling
4. File I/O operations (working with CSV, Excel, JSON, and text files)
5. Basics of modular programming (importing custom Python files/modules)
6. Using virtual environments and package management (pip)
7. Familiarity with Jupyter notebooks and script-based workflows 

=>Python's Pandas (Basic to Mid-Level-— Very Important):

1. Reading data from CSV, Excel, JSON, and databases
2. Creating, indexing, slicing, and filtering DataFrames
3. Applying groupby(), pivot(), pivot_table() for aggregation
4. Handling missing data: fillna(), dropna(), isnull()
5. Merging, joining, and concatenating DataFrames
6. Applying custom functions using apply() and lambda
7. Data type conversions and column renaming
8. Working with datetime fields and string operations
9. Writing data back to file or database

=>Python's NumPy (Basic to Mid-Level— Very Important):

1. Creating and reshaping arrays (reshape, flatten, transpose)
2. Element-wise operations and broadcasting
3. Indexing, slicing, and conditional selection
4. Aggregations: mean, sum, max, min, std
5. Random number generation (np.random)
6. Stack/split arrays (vstack, hstack, split)
7. Basic linear algebra: dot product, matrix multiplication (optional)

Last and most important concept- How to write, secure and optimized code, queries using Python and SQL.