Category Archives: Home

Python CSV

Working with csv files in Python

csv3

This article explains how to load and parse a CSV file in Python.

First of all, what is a CSV ?
CSV (Comma Separated Values) is a simple file format used to store tabular data, such as a spreadsheet or database. A CSV file stores tabular data (numbers and text) in plain text. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format.

For working CSV files in python, there is an inbuilt module called csv.

Reading a CSV file

# importing csv module 
import csv 

# csv file name 
filename = "aapl.csv"

# initializing the titles and rows list 
fields = [] 
rows = [] 

# reading csv file 
with open(filename, 'r') as csvfile: 
	# creating a csv reader object 
	csvreader = csv.reader(csvfile) 
	
	# extracting field names through first row 
	fields = next(csvreader) 

	# extracting each data row one by one 
	for row in csvreader: 
		rows.append(row) 

	# get total number of rows 
	print("Total no. of rows: %d"%(csvreader.line_num)) 

# printing the field names 
print('Field names are:' + ', '.join(field for field in fields)) 

# printing first 5 rows 
print('\nFirst 5 rows are:\n') 
for row in rows[:5]: 
	# parsing each column of a row 
	for col in row: 
		print("%10s"%col), 
	print('\n') 

The output of above program looks like this:

csv1

The above example uses a CSV file aapl.csv which can be downloaded from here.
Run this program with the aapl.csv file in same directory.

Let us try to understand this piece of code.

  • with open(filename, ‘r’) as csvfile: csvreader = csv.reader(csvfile)Here, we first open the CSV file in READ mode. The file object is named as csvfile. The file object is converted to csv.reader object. We save the csv.reader object as csvreader.
  • fields = csvreader.next()csvreader is an iterable object. Hence, .next() method returns the current row and advances the iterator to the next row. Since the first row of our csv file contains the headers (or field names), we save them in a list called fields.
  • for row in csvreader: rows.append(row)Now, we iterate through remaining rows using a for loop. Each row is appended to a list called rows. If you try to print each row, one can find that row is nothing but a list containing all the field values.
  • print(“Total no. of rows: %d”%(csvreader.line_num))csvreader.line_num is nothing but a counter which returns the number of rows which have been iterated.

Writing to a CSV file

# importing the csv module 
import csv 

# field names 
fields = ['Name', 'Branch', 'Year', 'CGPA'] 

# data rows of csv file 
rows = [ ['Nikhil', 'COE', '2', '9.0'], 
		['Sanchit', 'COE', '2', '9.1'], 
		['Aditya', 'IT', '2', '9.3'], 
		['Sagar', 'SE', '1', '9.5'], 
		['Prateek', 'MCE', '3', '7.8'], 
		['Sahil', 'EP', '2', '9.1']] 

# name of csv file 
filename = "university_records.csv"

# writing to csv file 
with open(filename, 'w') as csvfile: 
	# creating a csv writer object 
	csvwriter = csv.writer(csvfile) 
	
	# writing the fields 
	csvwriter.writerow(fields) 
	
	# writing the data rows 
	csvwriter.writerows(rows)

Let us try to understand the above code in pieces.

  • fields and rows have been already defined. fields is a list containing all the field names. rows is a list of lists. Each row is a list containing the field values of that row.
  • with open(filename, ‘w’) as csvfile: csvwriter = csv.writer(csvfile)Here, we first open the CSV file in WRITE mode. The file object is named as csvfile. The file object is converted to csv.writer object. We save the csv.writer object as csvwriter.
  • csvwriter.writerow(fields)Now we use writerow method to write the first row which is nothing but the field names.
  • csvwriter.writerows(rows)We use writerows method to write multiple rows at once.

Writing a dictionary to a CSV file

# importing the csv module 
import csv 

# my data rows as dictionary objects 
mydict =[{'branch': 'COE', 'cgpa': '9.0', 'name': 'Nikhil', 'year': '2'}, 
		{'branch': 'COE', 'cgpa': '9.1', 'name': 'Sanchit', 'year': '2'}, 
		{'branch': 'IT', 'cgpa': '9.3', 'name': 'Aditya', 'year': '2'}, 
		{'branch': 'SE', 'cgpa': '9.5', 'name': 'Sagar', 'year': '1'}, 
		{'branch': 'MCE', 'cgpa': '7.8', 'name': 'Prateek', 'year': '3'}, 
		{'branch': 'EP', 'cgpa': '9.1', 'name': 'Sahil', 'year': '2'}] 

# field names 
fields = ['name', 'branch', 'year', 'cgpa'] 

# name of csv file 
filename = "university_records.csv"

# writing to csv file 
with open(filename, 'w') as csvfile: 
	# creating a csv dict writer object 
	writer = csv.DictWriter(csvfile, fieldnames = fields) 
	
	# writing headers (field names) 
	writer.writeheader() 
	
	# writing data rows 
	writer.writerows(mydict) 

In this example, we write a dictionary mydict to a CSV file.

  • with open(filename, ‘w’) as csvfile: writer = csv.DictWriter(csvfile, fieldnames = fields)Here, the file object (csvfile) is converted to a DictWriter object.
    Here, we specify the fieldnames as an argument.
  • writer.writeheader()writeheader method simply writes the first row of your csv file using the pre-specified fieldnames.
  • writer.writerows(mydict)writerows method simply writes all the rows but in each row, it writes only the values(not keys).

So, in the end, our CSV file looks like this:

csv2

Important Points:

  • In csv modules, an optional dialect parameter can be given which is used to define a set of parameters specific to a particular CSV format. By default, csv module uses excel dialect which makes them compatible with excel spreadsheets. You can define your own dialect using register_dialect method.
    Here is an example:
     csv.register_dialect(
    'mydialect',
    delimiter = ',',
    quotechar = '"',
    doublequote = True,
    skipinitialspace = True,
    lineterminator = '\r\n',
    quoting = csv.QUOTE_MINIMAL)

Now, while defining a csv.reader or csv.writer object, we can specify the dialect like
this:

csvreader = csv.reader(csvfile, dialect='mydialect')
  • Now, consider that a CSV file looks like this in plain-text:
    We notice that the delimiter is not a comma but a semi-colon. Also, the rows are separated by two newlines instead of one. In such cases, we can specify the delimiter and line terminator as follows:csvreader = csv.reader(csvfile, delimiter = ';', lineterminator = '\n\n')

Data Types

Python | Set 3 (Strings, Lists, Tuples, Iterations)

Last Updated: 20-10-2020

In the previous article, we read about the basics of Python. Now, we continue with some more python concepts.

Strings in Python 
A string is a sequence of characters. It can be declared in python by using double-quotes. Strings are immutable, i.e., they cannot be changed.

Assigning string to a variable
 a = "This is a string"
 print (a) 

Lists in Python 
Lists are one of the most powerful tools in python. They are just like the arrays declared in other languages. But the most powerful thing is that list need not be always homogeneous. A single list can contain strings, integers, as well as objects. Lists can also be used for implementing stacks and queues. Lists are mutable, i.e., they can be altered once declared.

# Declaring a list 
L = [1, "a" , "string" , 1+2] 
print L 
L.append(6) 
print L 
L.pop() 
print L 
print L[1] 

The output is :  

[1, 'a', 'string', 3]
[1, 'a', 'string', 3, 6]
[1, 'a', 'string', 3]
a


Tuples in Python 
A tuple is a sequence of immutable Python objects. Tuples are just like lists with the exception that tuples cannot be changed once declared. Tuples are usually faster than lists.

Example

Create a Tuple:

thistuple = ("apple", "banana", "cherry")
print(thistuple)

Control Flow

Python if else

There comes situations in real life when we need to make some decisions and based on these decisions, we decide what should we do next. Similar situations arises in programming also where we need to make some decisions and based on these decisions we will execute the next block of code.

Decision making statements in programming languages decides the direction of flow of program execution. Decision making statements available in python are:

  • if statement
  • if..else statements
  • nested if statements
  • if-elif ladder
  • Short Hand if statement
  • Short Hand if-else statement

if statement

if statement is the most simple decision making statement. It is used to decide whether a certain statement or block of statements will be executed or not i.e if a certain condition is true then a block of statement is executed otherwise not.

Here, condition after evaluation will be either true or false. if statement accepts boolean values – if the value is true then it will execute the block of statements below it otherwise not. We can use condition with bracket ‘(‘ ‘)’ also.

As we know, python uses indentation to identify a block. So the block under an if statement will be identified as shown in the below example:

if condition:               # Statements to execute if    # condition is true
if-statement-in-java

if- else

The if statement alone tells us that if a condition is true it will execute a block of statements and if the condition is false it won’t. But what if we want to do something else if the condition is false. Here comes the else statement. We can use the else statement with if statement to execute a block of code when the condition is false.
Syntax:

if (condition):
    # Executes this block if
    # condition is true
else:
    # Executes this block if
    # condition is false

Flow Chart:-

Input/Output

Developers often have a need to interact with users, either to get data or to provide some sort of result. Most programs today use a dialog box as a way of asking the user to provide some type of input. While Python provides us with two inbuilt functions to read the input from the keyboard.

  • input ( type your input here)
  • raw_input ( type your input here )

input ( ) : This function first takes the input from the user and then evaluates the expression, which means Python automatically identifies whether user entered a string or a number or list. If the input provided is not correct then either syntax error or exception is raised by python. For example:

  • Các nhà phát triển thường có nhu cầu tương tác với người dùng, để lấy dữ liệu hoặc cung cấp một số loại kết quả. Hầu hết các chương trình ngày nay đều sử dụng hộp thoại như một cách để yêu cầu người dùng cung cấp một số kiểu đầu vào.
  • Trong khi Python cung cấp cho chúng ta hai hàm có sẵn để đọc đầu vào từ bàn phím.
    • input (lời nhắc)
    • raw_input (lời nhắc) input ():
  • Hàm này trước tiên lấy đầu vào từ người dùng và sau đó đánh giá biểu thức, có nghĩa là Python tự động xác định xem người dùng đã nhập một chuỗi hay một số hoặc danh sách. Nếu đầu vào được cung cấp không chính xác thì một trong hai lỗi cú pháp hoặc ngoại lệ được đưa ra bởi python. Ví dụ :
Python program showing
 a use of input()
 val = input("Enter your value: ") 
 print(val) 

Work-life balance of Software Engineer in Germany

The issues around finding the balance between family life, private life and work are gaining increased attention in political and business circles in Europe and Germany.

A key issue for many workers is flexible working time in order to have a work-life balance. Negotiating a work/life balance can help enable parents (both men and women) to reconcile their work with their family lives and women in particular to participate in the labour market. Finding the right work-life balance can allow workers to take leave from work so that they can participate in education or training or take up an interest, hobby or leisure pursuit. This may mean that employees can reorganise their working lives and hours around shorter days, weeks, months or years.

German families tend to be small with only one or two children. The men are still quite often considered to be the head of the household, even though both the wife and husband work.

At the turn of the century few employees in Germany were given holidays. In 1902, the metal and brewing industries gave three days annual leave to their workers. It was not until 1974 that the old Federal Republic introduced the statutory minimum holiday of 18 working days which has now risen to a minimum of 24 days. Today most collective wage agreements provide for holidays of six weeks or more and most employers give  holiday pay.

National holidays

Germany has quite generous holidays in comparison to other European countries. There are more public holidays in Germany than in any other European country. On these days, banks and most shops are closed, including supermarkets. However, many restaurants remain open. Public transportation and other services are also available. Many shops and businesses are also closed on Carnival Rose Monday (Cologne and Rhine region), Christmas Eve and New Year’s Eve although these are not official holidays.

Overview of legal holidays:

New Year                
Epiphany                   
Good Friday              
Easter Monday               
Labour Day               
Ascension                  
Whit Monday             
Corpus Christi                       
Assumption Day        
Day of German Unity            
Reformation Day       
All Saints’ Day                       
Penance Day              
Christmas                  
St. Stephen’s Day
01.01
06.01   (celebrated in BW, BY, ST)
around March/April
around March/April
01.05
May
May
May/June (celebrated in  BW, BY, HE, NW, RP, SL)
15.08   (celebrated in  BY, SL)
03.10
31.10   (celebrated in  BB, MV, SN, ST, TH)
01.11   (celebrated in BW, BY, NW, RP, SL
21.11   (celebrated in SN)
25.12
26.12

(Those States where the public holiday applies are shown in brackets; if nothing is indicated the holiday applies to all of Germany.)

Working hours

Opening hours

In Germany, businesses and shops are not legally allowed to stay open as long as they please and there are strict regulations concerning opening and closing hours. The German federal law “Ladenschlussgesetz” (Shop Closing Law) together with individual regulations in different States controls opening hours. Thus supermarkets for example close at 22.00 at the latest and open before 9 a.m. or 10 a.m. On Sundays almost everything is closed with the exception of bakeries and petrol stations.

 Working times

The German Working Time Regulations (“Arbeitszeitgesetz”) regulate working hours on a legal basis. They are based on the European regulation 93/104/EG. In addition, most industries have collective agreements that regulate working hours and holidays. However, it can be said, that a working week of more than 48 hours on average during a  6 month period must not be exceeded. Furthermore, Sundays and national holidays are non-working days.

Working culture

Germans see themselves as modern, liberal and cultured, and working practices are formal and professional. The following outlines the working practices that you should be familiar with before investing in Germany:

  • Though long-term relationships are considered very important, friendships are usually not developed too quickly. It may take some time before personal names are used between non-familial parties.
  • German business culture has a well-defined and strictly observed hierarchy, with clear responsibilities and distinctions between roles and departments.
  • Professional rank and status in Germany is generally based on an individual’s achievement and expertise in a given field. Academic titles and backgrounds are important, conveying an individual’s expertise and thorough knowledge of their particular area of work.
  • An important aspect is Germany’s work ethic. Employees define themselves as part of the corporation they are working for and quickly identify themselves with its product and/ or services.
  • Rank is very important in business. Never set up a meeting for a lower ranked company employee to meet with a higher ranked person.
  • Notwithstanding what has been said previously, today over half of all university graduates are women. Female students are well represented in the professions; they lead in some fields such as medicine and law. The new availability of qualified female graduates is likely to bring great changes in the German workplace of the future.
  • Pay and power inequalities are still present however. Male employees tend to receive higher wages than their female counterparts. Jobs considered as being “women’s work” typically pay less than those deemed “men’s work”.
  • In more traditional companies, it is still generally true that everything is run by committees, things are discussed in great length and risk taking is not as common as in other countries.
  • There is one philosophy for almost everybody in German business: if someone says he is going to do something, he will do it. The same is expected of others as well. Never make a promise that you cannot keep or offer something that you cannot deliver. Germans dislike and do not trust unreliable people.
  • There is no legislated or administratively determined minimum wage. Collective bargaining agreements set minimum pay rates and are enforceable by law for an estimated 80 to 90 per cent of all wage and salary earners
  • Federal regulations limit the working week to a maximum of 48 hours, but collective bargaining agreements may supersede these. Contracts that directly or indirectly affect 80% of the working population regulate the number of hours of work per week.
  • The average working week is around 40 hours; rest periods for lunch are accepted practice. Provisions for overtime, holidays, and weekend pay vary depending upon the applicable collective bargaining agreement.
  • An extensive set of laws and regulations govern occupational health and safety. A comprehensive system of worker insurance enforces safety requirements in the workplace.

It is important that these issues are examined and understood before setting up a company and employing a workforce in Germany. These issues differ all over Europe but legal guidelines are set by the European Commission.

Health insurance

Germany’s health care system provides its residents with nearly universal access to comprehensive high-quality medical care and a choice of physicians. Over 90% of the population receives health care through the country’s statutory health care insurance programme. Membership of this programme is compulsory for all those earning less than a periodically revised income ceiling. Nearly all of the remainder of the population receives health care via private for-profit insurance companies. Everyone uses the same health care facilities.

Live*

This is a page to share my thoughts and feelings about new events, happening live today or in the near future, all I share is my personal thoughts, looking forward to discussing with everyone.

VIDEO] Live Stream Best Practices (1/2) | Believe

Đây là chuyên mục cập nhật thường ngày, nơi mình đưa ra những bài viết về những vấn đề nổi cộm hiện nay, tất nhiên là dưới góc nhìn của một lập trình viên kinh nghiệm đang làm việc tại 1 trong những công ty vận tải lớn nhất thế giới:

Nội dung chia sẻ bao gồm:

Lập trình

Khoa học máy tính

Thông tin kinh tế, chính trị xã hội

Hoặc các chủ đề khác nếu nhiều bạn thích nói về nó thì mình sẽ viết bài trên này hoặc là làm vlog youtube để nói về vấn đề đó , các bạn theo dõi kênh của mình tại đây nhé:

How to build Robot Project in Frankfurt University Germany – YouTube

Pointers & References

Pointers store address of variables or a memory location.

// General syntax
datatype *var_name; 

// An example pointer "ptr" that holds
// address of an integer variable or holds
// address of a memory whose value(s) can
// be accessed as integer values through "ptr"
int *ptr;  

Using a Pointer:

To use pointers in C, we must understand below two operators.

  • To access address of a variable to a pointer, we use the unary operator (ampersand) that returns the address of that variable. For example &x gives us address of variable x.
// The output of this program can be different 
// in different runs. Note that the program 
// prints address of a variable and a variable 
// can be assigned different address in different 
// runs. 
#include <stdio.h> 

int main() 
// Prints address of x 
printf("%p", &x); 

return 0; 
}
  • One more operator is unary * (Asterisk) which is used for two things :
    • To declare a pointer variable: When a pointer variable is declared in C/C++, there must be a * before its name.
// C program to demonstrate declaration of 
// pointer variables. 
#include <stdio.h> 
int main() 
{ 
    int x = 10; 
// 1) Since there is * in declaration, ptr 
// becomes a pointer varaible (a variable 
// that stores address of another variable) 
// 2) Since there is int before *, ptr is 
// pointer to an integer type variable 
int *ptr; 

// & operator before x is used to get address 
// of x. The address of x is assigned to ptr. 
ptr = &x; 

return 0; 
}
  • To access the value stored in the address we use the unary operator (*) that returns the value of the variable located at the address specified by its operand. This is also called Dereferencing.