Devops(Series) Phần 2 : AWS EC2, nhân tố quan trọng của AWS.

Aws EC2 là gì? Khi sử dụng Virtual Machine thì các bạn có thể dùng virtualbox, Vm ware, Vagrant, VM ware. Ngoài các tool phổ biến trên còn một số tool khác mà mình không liệt kê hết ở đây.

Một vitural machine cung cấp nhiều thứ ,nhưng quan trọng nhất là: CPU, RAM, và network.

Tương tự, trên các hệ thống cho thuê server như GCP(Google Cloud Platform), Microsoft Azure hoặc AWS, những VM này có các tên khác nhau, như trên Azure và Google Cloud là Compute, và trên Amazon Web Services là EC2(Elastic Compute Cloud).

Mặc dù tên gọi khác nhau nhưng về cơ bản EC2 cung cấp cho bạn 1 máy chủ với CPU, RAM, Network để bạn có thể làm việc, bạn biến nó thành 1 server hay chạy web services gì đó tùy theo bạn muốn, và EC2 nằm bên trong 1 VPC, với Gateway để kết nối internet, và ở gateway bạn có thể đặt cá rules để bảo vệ cho Instance của bạn.

Ngoài ra đối với những hệ thống thực tế, khi chạy webservices trên EC2 bạn có thể tùy chỉnh thêm autoscaling, tự động tăng giảm số lượng máy ảo khi server quá tải, hoặc tự điều chỉnh cắt giảm số lượng máy ảo trong thời gian vắng khách.

Learn from the experts: Create a successful blog with our brand new course

WordPress.com is excited to announce our newest offering: a course just for beginning bloggers where you’ll learn everything you need to know about blogging from the most trusted experts in the industry. We have helped millions of blogs get up and running, we know what works, and we want you to to know everything we know. This course provides all the fundamental skills and inspiration you need to get your blog started, an interactive community forum, and content updated annually.

7 Steps update your LinkedIn profile to get dream job in 2020.

2020 is a very special year, in this year we learn some new words, bad words: Lockdown, Covid-19, World-pademic, stay at home … It can be said that this is a sad year. But it is also an opportunity for us to give love to other people around us, while looking at ourselves and preparing for the good of 2021.

Branding is time and resource consuming , big companies are trained on it – and they’re good at it. But how do we often use those skills to build our own personal brands? For many of us, we don’t usually get involved in personal branding . And in this post I want to share the 7 steps I take to build personal branding on Linkedin, with hundreds of thousands of others, not just for a good job, but also for personal branding. mine.

We don’t because we are busy and because it can sometimes feel selfish or egotistical to invest time in marketing ourselves. But by ignoring personal brands, we don’t just sell ourselves – we miss a huge opportunity from a marketing perspective. The impact of those who share content is enormous. And the most effective employees sharing are the ones who have built their personal branding on LinkedIn.

Here are 7 profile features you should check out and update for 2020.

1. Choose the right profile picture for LinkedIn

Your profile photo is your business card on LinkedIn – that’s how people are presented to you and (visual creatures are us), it dominates their impression in the first place. There are some great posts explaining how to choose the right profile picture on LinkedIn – but here are some quick tips to get started: make sure that photo is recent and like you, makeup on your face about 60% (long – picture taken horizontally does not stand out), wear what you want to wear to work and smile with your eyes n.

2. Add a background image

Your background image is the second image element at the top of your profile page. It obtained the attention of everyone, put context and show a little more about what’s important to you. More than anything, the right background images help your site stand out, collect the attention and always memorable.

3. Set your title is not just a job title

There’s no rule saying that the description at the top of your resume page is just a job title. Use the title field to say a little more about how you see your role, why you do what you do, and what makes you tick. If you have sales reps at your company who are engaged in social selling, then take a quick look at their profile page titles for inspiration. They will almost certainly have more of their job titles in it.

4. Turn your summary into your story

The first thing to say about your LinkedIn summary is – make sure you have one . Your summary is your chance to tell your own story – so don’t just use it to list your skills or job titles you already have. Try to make it descriptive about why those skills are important – and the difference they can make for the people you work with. Don’t be afraid to invest time, try a few drafts and run your summary in front of people you know. This is your most personal piece of content marketing , so speak your own language .

5 . Grow your network

Take advantage of the LinkedIn feature that suggests people you can connect with. It’s amazing how effective this can be at finding relevant people to reach you on , no connection requests being sent without your permission, because So you can check all potential connections. Also, get into the habit of chatting with the LinkedIn connection requirements – it’s a great way to keep your network up and running.

6. Share content related to your work

Your LinkedIn Have a network of connections on LinkedIn, and you have an active role in that network, appearing in the LinkedIn feed of your connections in a way that adds value to them. Sharing relevant content with your network is one of the most accessible ways to do this. You can start by tracking information on linkedin of themselves and share content that you find really interesting position , or related to your industry .

7. Publish long-form content – and use it to initiate a conversation

The more you share and comment on content, the more you will establish your expertise and thought leadership information on LinkedIn. Publish post long form is the next step according to the natural need to take. A great starting point is to track the response you get to your comments and shares. Are there specific topics and perspectives that seem relevant to your network? Is there a comment you shared that you feel would be expandable in a post? Developing your thought leadership in this way keeps it realistic – and keeps you on the lookout for the issues your relationships are talking about. Get ready for your long posts to start new chats. Keep track of comments and be ready to respond.

Make your LinkedIn profile more active so that you don’t have to waste time organizing your resume for a job, getting the recruiter to find you . Try working through these ideas, building from idea to idea – and you’ll find that you can make rapid progress, even if you can only spend a few minutes in lunch break or in the evening. After taking full advantage of your LinkedIn profile, you’ll be amazed at the difference it can make for both you and your business . 

Let 2021 be a new start for you!

Photo by Nicole Michalou on Pexels.com

5 điều cần biết trước khi viết blog hoặc website! (SEO)

Nói nôm na đây là bài viết về các mẹo làm sao để bài viết của bạn được lên top tìm kiếm của Google. Mặc dù có rất nhiều cách nhưng thứ giữ lại khán giả vẫn là nội dung các bạn nhé, nào bắt đầu vào đề thôi (-_-) .

1 – Hiểu biết về SEO: (Tối ưu hóa Công cụ Tìm kiếm) là một trong những yếu tố quan trọng mà mọi blogger nên cân nhắc trước khi pos bài. Bạn muốn bài viết của bạn có nhiều người xem thì phải làm sao nó xuất hiện nhiều trên Google. Nhiều website không quan tâm đến SEO; họ chỉ cần chọn một themes thân thiện google và bắt đầu post bài. Đây cũng chính là lý do họ bị phụ thuộc vào các thuật toán của Goolge như Google Panda và Penguin.

Photo by Tobias Dziuba on Pexels.com

Xếp hạng công cụ tìm kiếm cao hơn không chỉ phụ thuộc vào chủ đề bài viết của bạn, mà còn ở tiêu đề của các bài đăng trên blog của bạn. Nếu bạn có một blog hơn 100+ bài viết, một số bài viết trong blog của bạn có thể phổ biến và một số trong số đó có thể không phải là một bài viết phổ biến. Bạn đã bao giờ nghĩ tại sao chỉ có một vài bài viết được xem nhiều trong blog của bạn? Bởi vì dù có cố ý hay không, vô tình bài viết nhiều view của bạn có tiêu đề phù hợp với công cụ tìm kiếm như là google, hoặc bạn cung cấp nội dung ít người tìm kiếm.

2- Một nguyên tắc chung là: Từ khoá của bạn phải nằm trong tiêu đề bài đăng và hầu hết người viết mắc lỗi khi chỉ thêm Từ khoá và không quan tâm đến việc tối ưu hoá nó để có CTR(tỷ lệ nhấp chuột) tốt hơn. Có Từ khóa trong tiêu đề bài đăng của bạn sẽ giúp ích rất nhiều, nhưng khi nói đến việc tăng CTR, cách bạn đặt các tiêu đề bài đăng của bạn tạo ra sự khác biệt rất lớn.

Ví dụ hãy xem 2 cách đặt tiêu đề sau:

Làm sao đưa bài này lên top (Tiêu đề trung bình)

Điều duy nhất bạn cần làm để đưa bài này lên top (Tiêu đề sẽ giúp bạn nhận được nhiều CTR hơn)

Như mình đã nói, tiêu đề thân thiện với SEO rất quan trọng để xếp hạng công cụ tìm kiếm tốt hơn, tất nhiền là bài viết của bạn cũng phải hấp dẫn, nếu không …… thì sẽ không giữ được người đọc.

Trong bài viết này, mình sẽ chia sẻ cho bạn “Cách viết tiêu đề có thể chơi với công cụ tìm kiếm không chỉ mang lại cho bạn nhiều truy cập miễn phí mà còn cải thiện tỉ lệ nhấp chuột lên trang web của bạn“. Tiêu đề ví dụ mà mình đã chia sẻ ở đây là “Làm thế nào để có thêm lưu lượng truy cập blog”. Tiếp tới đây, mình sẽ đặt tiêu đề này làm tiêu đề thân thiện với công cụ Tìm kiếm.

Các bước để viết tiêu đề chơi được với SEO:

Nếu bạn chưa quen với SEO, mình xin nhắc lại một lần nữa là từ khóa của bạn phải là một phần của tiêu đề bài post của bạn. Bạn có thể tham khảo hướng dẫn Harsh về cách viết tiêu đề bài đăng khác nhau cho người đọc và Công cụ tìm kiếm. Một mẹo mà tôi muốn đưa ra ở đây, hãy luôn chọn tiêu đề bài đăng của bạn sau khi hoàn thành bài viết, và hãy suy nghĩ kỹ cho chủ đề bài viết.

3- Phân tích từ khóa của bài viết Phân tích từ khóa là bước đầu tiên để làm cho tiêu đề bài đăng của bạn thân thiện với SEO. Có nhiều công cụ phân tích từ khoá có sẵn trực tuyến, chẳng hạn như công cụ phân tích từ khoá của Google. Hãy mình phân tích các tìm kiếm từ khóa trong bài viết của mình trên Google nhé. Đầu tiên mình sẽ viết một bài báo, cụ thể là “Làm thế nào để có thêm lưu lượng truy cập blog”. Đầu tiên tôi Mở công cụ Từ khoá của Google Adwords(như là Semrush) và kiểm tra các trang khác, Số lượng tìm kiếm… Vv… của từ khoá chính trong bài viết của mình. Sau khi Phân tích, mình nhận thấy rằng danh sách tìm kiếm từ khóa “Cách tạo lưu lượng truy cập blog” nhiều hơn và có mức độ cạnh tranh thấp. Bây giờ mình đã tìm thấy từ khóa chính để chèn vào tiêu đề của mình. Nếu bạn là một trong những người thích Từ khóa đuôi dài, tôi khuyên bạn nên thử SEMRUSH hoặc Ahrefs vì công cụ này giúp bạn tìm Từ khóa đuôi dài dựa trên từ gốc của bạn.

4- Sử dụng các từ khóa mục tiêu chính cho tiêu đề của bạn: Bây giờ mình đã tìm thấy từ khóa mà tôi nên chèn nó vào tiêu đề bài đăng của mình. Đầu tiên, mình đã nói điều đó, tôi sẽ giữ tiêu đề của mình là “Cách để có thêm lưu lượng truy cập blog”, sau khi tôi chèn từ khóa chính vào tiêu đề của Bài đăng và sau khi thực hiện một số thay đổi trong tiêu đề, tôi đã thực hiện nó như “Mẹo tạo lưu lượng truy cập để tăng lưu lượng truy cập vào blog của bạn”. Tiêu đề mà tôi đã tạo cũng thân thiện với công cụ tìm kiếm và hấp dẫn, loại tiêu đề hấp dẫn này khiến mọi người háo hức mở bài viết của bạn và đọc nó. 80% của phần đã kết thúc. Sau đó, những gì về 20% còn lại? , bạn có thể tìm thấy nó ở bước 3!

5- Bước thêm số: Rất quan trọng! 80% phần làm cho tiêu đề bài viết thân thiện với SEO đã kết thúc! Bây giờ là lúc để thêm phần hoàn thiện vào tiêu đề của bài viết! 20% tiếp theo là thêm một số hoặc tiền tố-hậu tố, vào tiêu đề của bạn để cải thiện CTR hơn nữa. Ví dụ trong bài này mình trình bày 5 cách để nhận được nhiều view cho blog hơn. Và vì thế nên có số “5” trên tiêu đề của bài post. Và tiêu đề là: “5 điều cần biết trước khi viết blog hoặc website!”

Bây giờ mình đã tạo một tiêu đề thân thiện với công cụ tìm kiếm cho bài đăng trên blog của mình và điều đó sẽ thu được nhiều người xem hơn mình thường, không biết có bài này có giúp mình giàu lên không:))!

Bài viết đến đây kết thúc:). Hy vọng các bạn có thể áp dụng điều này vào blog của mình nhé.

Chào Thân ái!

Devops chuyện chưa kể(Series) phần 1 : Truyền thuyết Amazon Web Servies (AWS ).

Truyền thuyết kể rằng:

Năm thứ nhất sau Covid-19, anh Tèo, CTO của lele.com, 1 trang web chuyên về gà, đang gặp rắc rối nghiêm trọng về tương lai công ty…

Công ty anh điều hành 1 website gọi gà ở chợ đầu mối, như những web gọi gà khác, trang web của anh có cấu trúc như sau:

Khách hàng sẽ vào trang web của tèo tìm gà, ưng ý bấm chọn, thì browser sẽ gửi request tới server để lấy thông tin của gà được chọn thông qua domain name server( từ đây mình sẽ viết tắt là DNS), DNS này dùng để phiên dịch tên miền lele.com thành IP cho máy dễ hiểu thôi. Sau khi gửi request syn ack các kiểu, giao thức TCP-IP được thành lập thì server sẽ gửi hình của gà cho các bạn xem, và nếu ok sẽ tiến hành bước tiếp theo là đá gà.

Ok mọi chyện sẽ khá bình thường cho đến 1 ngày trang web trở nên nổi tiếng vì dàn gà mới đi thi đoạt giải, dẫn tới số lượng user tăng lên vùn vụt, mỗi ngày có thêm 10 ngàn user, khách nhiều thì tốt nhưng nhiều quá cũng không tốt đẹp gì…Và trang của anh gặp phải vài vấn đề đau đầu:

  1. Khi quá nhiều lượt truy cập vào 1 server, băng thông sẽ bị nghẽn và làm chậm tốc độ load web.
  2. Số lượng gà mới có profile khủng hơn, cty anh tèo phải cung cấp thêm nhiều thông tin vì khác hàng yêu cầu, dẫn tới sever quá tải. Đầu tư thêm server thì mất công setup network này nọ. Mà upgrade thì web vẫn phải chạy để níu chân khách.
  3. Mặc dù khách đông nhưng thường chỉ hoạt động về đêm làm lag server, có những thời điểm web không có khách. Nên đầu tư thêm để ban đêm phí quá.

Nhiều vấn đề không được giải quyết, Tèo quyết định thuê chuyên viên về thay đổi cấu trúc cty. Cuối cùng họ quyết định: Đưa tất cả lên Cloud!

Nhưng có nhiều dịch vụ cung cấp cloud thì chọn cái nào? Xem lại trang web cạnh tranh của Tèo là xemheo.com họ cũng đã chuyển hết Server, cơ sở dữ liệu của họ cho Microsoft Azure. Tèo quyết định không chơi đụng hàng, sau 1 một hồi phân vân và anh chọn AWS vì AWS có giao diện dễ sử dụng hơn:

Amazon Web Services: cung cấp một loạt các sản phẩm dựa trên đám mây trên toàn cầu bao gồm máy tính, lưu trữ, cơ sở dữ liệu, phân tích, mạng, thiết bị di động, công cụ dành cho nhà phát triển, công cụ quản lý, IoT, bảo mật và các ứng dụng doanh nghiệp: theo yêu cầu, khả dụng trong vài giây, trả phí -giá cả tùy ý. Từ kho dữ liệu đến các công cụ triển khai, thư mục đến phân phối nội dung, hơn 175 dịch vụ AWS có sẵn.

Anh Tèo quyết định trở thành khách của AWS. Anh đăng kí mua gói cơ bản, giờ anh có 1 server chạy trên AWS, services thì chạy trên EC2, các backend method đã có Lambda, dữ liệu thì lưu trữ trên S3 bucket, anh còn đang nghiên cứu chuyển qua dùng Kafka cho giống Linkedin vì kafka dữ liệu cập nhật và lưu trữ thông minh hơn. Service của anh giờ còn có chatbot để chat cùng khách hàng và anh được theo dõi toàn bộ server của mình thông qua Amazon Cloud Watch.

Với AWS, Tèo đã phát triển web bán gà của anh thành một thương hiệu International, với quy mô đàn gà 1000 con và nhiều gà ngoại.

Tèo đã xem video sau và thành công, còn bạn thì sao?:

Author:

NGUYEN TRUONG THANH / Software Engineer in Atomotive, Germany

_________________________________________

E-mail: thanhnguyen1181991@gmail.com  

Phone: 004917657997009

GLS it Services, Eschborn, Frankfurt am Main, Germany 

Website: thanhnguyensite.net

Using AI for object classification.

In this post I will show you the easiest way to combine AI, convolution neural network(CNN) and docker container to classified object in real time. So all thing you need to know is basic knowledge about docker and neural network. If you are very new to programming, don’t worry, just follow the step below, and you will have a program classified object in real time.

in the video above I’m driving a car go around with a camera on top, to tracking other car and person inside it. I use CUDA Yolo + Nvidia GPU. You can also do the same, all you need to do is download my Docker file and run it.

For who need to understand the theories behind, I will summaries like this. The docker file will create a Ubuntu Linux environment and install Nvidia GPU+OpenCV+darknet in to it. Darknet is a wonderful neural network, it was train by around 10 millions picture and can real-time recognize about 70 categories (car, dog, cat, ship, plane….). If you want to learn more about darknet, you can read my article : https://thanhnguyensite.net/2020/11/05/neural-network/

OK! now let’s go the AI world:

Darknet Nvidia-Docker Ubuntu 16.04

Prerequisites

  1. Make sure you have the NVidia driver for your machine

Find out your the Graphics Card model

lspci | grep VGA

https://www.nvidia.com/Download/index.aspx?lang=en-us

How to install NVidia Drivers on Linux https://gist.github.com/wangruohui/df039f0dc434d6486f5d4d098aa52d07#install-nvidia-graphics-driver-via-runfile

  1. Install Docker and NVidia Docker https://github.com/NVIDIA/nvidia-docker

Steps to run

  1. Clone this repo:
git clone https://gitlab.com/thanhnguyen1181991/darknet-docker.git
  1. Build the machine (this step might take a while, go make some coffee)
docker build -t darknet .
  1. On start.sh make sure you have the correct address of your webcam, in file start.sh line 8, if you use laptop onboard webcam, then choose: device=/dev/bus/usb/003/004:/dev/video0, if use external webcam, then: device=/dev/bus/usb/003/004:/dev/video0

Find your webcam bus

lsusb -t

Change the following line with the correct webcam bus

--device=/dev/bus/usb/003/002:/dev/video0
  1. Map a local folder to the Docker Container

Format:

/local/folder:/docker/folder

on start.sh change the following line

-v /home/projects:/dev/projects \
  1. Run the machine with Webcam
sh start.sh

Darknet

Make sure you have the weights for what you want to run

More information at https://pjreddie.com/darknet/

6 tips for Linux noob (like me)

In this post I will share experience using linux command, and was we can do and play with a operating system for Developer, or shorter is Linux. You don’t need to know more about this, I will show you how you can create funny program with some Linux command, what you can’t do in Windows.

1. Linux updates and upgrades do not require you to reboot

Re-booting after every software installation or update is very annoying in Windows. I keep wondering why this is not necessary in Linux, but on Windows is rule of tumb – every installation is asking you to re-boot, or after you download the Upgrades annoying window pop up and say – will reboot in 10 minutes, save your work. This is really annoying.

With Linux all you need to do is run : sudo apt update

Then have a cup of coffee until it update your computer.

2. No need to install drivers every time you plug in your computer USB device

OK, I understand that there are custom devices which use uncommon drivers like printers, cameras, etc, but why on earth every time you plug in Windows simple mass storage disk drive, or USB-Serial converter, or Mouse, or Keyboard – devices which are standard and are embedded in every Windows OS after W98 it ALWAYS ask for drivers which are already there? Why on Linux I plug the external HHD or Flash drive and it automatically mounts on my computer and I can work with it, while Windows is asking me for drivers and several minutes scans and show me different windows with warnings and Continue buttons like I’m doing something scaring which may ruin my OS?

3. You can move image/boot-able drive between machines without need for reinstall

Yes! this is something windows users can’t imagine is possible! I do remember back in the dark ages Windows asked me to re-install after I have upgraded the RAM memory size! Now imagine you get your Windows boot-able HDD and plug it on other computer, will it boot? no way!

4. system config in files not registers mess

Now this is one of the most annoying WIndows features – after several months of installing and removing software your registers and windows/system directory becomes so bloated with shared DLLs and mess that some people start making money by writing registry cleaning software!

5. You can’t boot windows from USB Stick

Probably they didn’t find a way to ask you for registration and to collect your money every time they sense this USB is plug to other computer??

6. You cant see this on Windows:

# uptime
15:54PM up 122 days, 11:22, 5 users, load average: 0.12, 0.30, 0.13

every few days if you do not reboot windows machine it starts to act slowly due to the severe memory fragmentation

An overview about Metal as a Services(MaaS)

Hello again! Today’s class is metal as a service MAAS so we’ve been doing a number of classes lately on these different services, we’ve talked about software as a service we’ve talked about infrastructure as a service, and we have talked about platform as a service. Now when you’re talking about these services by far the most popular solution out there is software as a service that is where you basically you go out and you lease software from companies and the software is all run on their server so we’re talking about software as a service think about things like Google Docs think about things like Salesforce the software is not installed on your local computer.

Its installed on their servers and you access that somehow these are either through a web browser or through some type of a terminal connection something like that tied by the infrastructure as a service basically when your your infrastructure all those things that you would buy and install in your premises you no longer actually own anymore.

So think about your telephone systems companies you should spend $50 thousand on their telephone system and that telephone system was installed in their premises and they owned it now you can get hosted voice over IP solutions such as on sip you get hosted firewall solutions you can have hosted server rooms why have your own servers when you can go to Amazon Web Services and simply spin up a number of virtual services on their platform so that is infrastructure as a service.

We then talked about platform as a service platform as a service is where you create your web apps and then you are looking for some place to host them so the basic idea is think about a shared hosting web plan you create your web site you create your web application and then you simply upload that to GoDaddy or one in one servers their servers have PHP installed their servers have my sequel installed their servers have Ruby or Perl or any of these other scripting languages that you need all that you have to worry about is your application of course that gets much more complicated once you go over to Google App Engine and some of the more and more advanced things but that’s the the basic concept so now we’re getting to basically probably the last service that I will be talking about is metal as a service now this is one of those really really really cool ideas that actually it is pretty cutting-edge I’m not sure if it’s bleeding edge but it’s pretty cutting edge so some of you guys looking to create businesses out there really should listen to what I’m talking about right now because it really is a good business opportunity right now because not very many people actually offer this service so when we’re talking about metal as a service what we are literally talking about is providing the server hardware as a service to clients so this is not the same thing as what you would normally think about with cloud computing or virtualization where you install a hypervisor onto a server and then they spin up some kind of a instance of a server and this isn’t the same thing as a dedicated server so for thanhguyensite.net and a couple of other things that I do we rent a dedicated server from a company called one and one comm with that I get a specific server 12 gigs of ram quad core processor blah blah blah.

With a certain version of Linux on it and then from that point I can configure up but with that when you are purchasing something like a dedicated server you have to use whatever operating systems are provided by the provider so with one and one com if you go with them you can use like Server 2008 or Server 2012 Fedora sent OS if you wanted to use something else tough luck if you wanted to use freebsd on one of these services that you couldn’t do it if you wanted to install a hypervisor on Twenties dedicated services servers you couldn’t do it the reason is is because although you’re renting the dedicated server it has to have a bare minimum installation on it before you are actually able to get access to it the cool thing with metal as a service it is the concept is that you are literally renting the physical box with nothing else on it so this is where you would go to a provider and you would literally rent it would be a quad-core xeon processor with so much RAM so much hard drive and that’s it there would not necessarily be sent OS on it there were not necessarily be windows on it there wouldn’t necessarily be anything on it.

You are literally renting the metal as a service so basically now instead of having to have your own server rooms with your own equipment your own server racks your own HVAC you’re all redundant to power supply and your ups and all of that kind of stuff you can have the same thing sitting in somebody else’s server room so they are renting to you the metal as a service now that you may be wondering why why would you bother with that if you can get virtual servers if you can use infrastructure as a service if you can even get dedicated servers why would you run want to rent or lease the bare metal as a service well as you go through with your companies.

If you have a startup company or if you have a technology company and you start growing what, you are going to find is no matter what operating system distribution you use. It is probably not going to be optimized for whatever it is you’re doing so you know we all know with Windows we all know with Windows. Windows hogs up a lot of extra resources to do things most of us really truly don’t care about it we’ll be happy if it did it but a lot of people don’t realize is even with Linux.

Even with Linux there are resources that are used there are security vulnerabilities that are opened up simply because when you install the default installation of whatever Linux you’re going to be using it installs a base level of applications and services and a lot of times you don’t need that so imagine if you were a company where you want to spin out a lot of database servers and you want those database servers to run at the absolute optimum the fastest they can possibly run well you may want to go in to a distribution of an operating system and literally rip out all the crap that you don’t want you don’t need notepad you don’t need tar you don’t need a lot of these these things you just need that server to run as fast as possible to do a specific task possibly do something like a database server because this becomes very important especially when you start dealing with larger companies that are dealing with a larger load on their servers because when you rip out all of the crap you don’t need on it on a servers operating system.

You can gain efficiencies now this is not you’re not going to probably speed up the server by 200% or 500% or a thousand percent right that’s not what what the target you’re going to hit you may be able to actually speed up the server though by something like five percent now for you especially if you haven’t dealt with real server rooms if you haven’t dealt with real loads on servers optimizing an operating system we get five percent improvement probably doesn’t sound like a big deal but with companies if you if you have 20 servers up and running or 40 servers up and running a five percent increase in speed can be very very very very very very very significant so with these companies they may be looking to optimize the things like I say the operating systems that will be installed how all this will be configured how all this will be managed and so all they want is the bare metal they want the server they want the hard drives they want the RAM.

They want the CPU but they don’t want anybody else to mess with the rest of it they want to be able to build this thing from the ground up and again there can be a lot of reasons for this nowadays things like again efficiency making sure that the resources on the server are optimized but also issues such as compliance so compliance is becoming a bigger and bigger deal within the the corporate world what compliance means is that you are running your IT systems to certain specifications for security and reliability so as more and more companies start using cloud computers and servers and all that to run the infrastructure of their business they have to make sure that that infrastructure is reliable enough for their industry now one of the problems if you go out and you use a standard instance of an operating system or you use a standard load of an operating system from one of these providers is you don’t necessarily know all the security flaws you haven’t necessarily been able to sit down and do penetration test and do hardening testing and do all of those things so when you when you are leasing let’s say from 1 + 1 , dedicated server you can’t guarantee that this is that the the server operating system that has been installed is as hardened as it should be.

Now again for somebody like me I don’t care again do good backups and you should be fine for and this is one of the things you have to think about for 98% of the business population this type of concept doesn’t matter but for that 2% it is very very very important it is very important that they know that whatever operating system and software that’s going to be installed in that server it lives up to certain specifications so that’s why they would want to be able to rent that that bare metal as a service so metal as a service now one of the questions that you’re going to be coming up in with thanh’s you’re gonna be saying work and got me saying.

Well Thanh uh I don’t understand how you would interact with metal as a service then because you know when we think about dealing with these virtual computers when we think about dealing with it with cloud computers and all that we have a basic interface to deal with so basically again if you get a dedicated server you get a virtual private server they spin up the operating system and then they give you the login credentials so basically the company that you’re buying the service from they have already installed the instance of the operating system they’ve already created the first user account they then give you that information for that first user account and then you can figure it out.

However it is you want so the question you may be asking say well I don’t I don’t understand that because if you’re literally renting the metal and the metal let’s say five states away well wait a minute but there’s no operating system to interact with and the metals five states away so you don’t want to drive there so so I don’t understand how you would configure it or work with it well one of the cool things and not really new but but they’re they’re coming too more into vogue is something called IPP KVM switches so kate.

KVM switches been around for forever long far longer than i’ve been in the computer industry keyboard video mouse switches so what these are generally when you’re dealing with a server rack is you plug all the servers in the rack into one KVM switch and then you can press a button then when you press that button that gives you access to the server from one keyboard video and mouse combination so you have a monitor you have a keyboard and mouse and you say oh I need to deal with the wit server – and you just hit the server – button and server – pops up oh I need to deal with server 10 you click the server 10 button and the server 10 pops up well with KVM they now have IP k via what this means is that you can deal with that server from the basic input/output so the basic video keyboard and mouse and you can do that over an IP connection so you can either open that up through a web browser and be able to log in or you can open it up through some kind of terminal session or or some kind of application so basically you can be sitting in your in your office five states away from this bare metal the company that you’re dealing with will plug in the KVM switch and whatever else and then basically you can hit the on there.

They can hit the on button or you may have some kind of remote wait at the on button and from there it will literally load into a BIOS screen then from there depending on what the the service provider has for you you can go and you can install your your your your operating system and do all of your configurations but literally you have remote access to the lowest level of that server so you could literally reboot that server and go into the BIOS and change BIOS configuration settings you literally have that ability whether you’re five states away or you know on an entirely different continent now especially with PDUs so the the power distribution units basically what most people will call surge protectors even those have remote access so that you can do things like power cycle the server because again the question where you’re like well Thanh I don’t understand if if you have metal as a service.

If you have that metal and you do something wrong and it freezes up how do you force it to restart because again you know you’re installing operating systems you’re doing all kind of wacky stuff sometimes it’ll freeze and if you have access to the metal what are you going to do well with these surge protectors these power distribution units you can actually power cycle them again remotely so this is the cool stuff with metal as a service I think this is going to become a much more prominent thing right now this is one of those things that it is offered by companies you don’t see it around a lot but it is something that you should be looking at and you should be thinking about because again this way you can have you you can have your own custom servers that have been hardened up to your specifications but they are sitting in somebody else’s data center you don’t have to worry about it just like with all these commoditized items it is less expensive for you to be able to rent this service from somebody else that can have a thousand or five thousand or twenty thousand of these servers up and running basically they can have five or ten technicians running around making sure all the metal is doing with metal supposed to do versus if you had servers in each one of your individual offices and having people run around and deal with that kind of stuff so that’s the basic concept of metal as a service again all it is at the end of the day is you are literally leasing or renting that bare metal so you’ve got a server with absolutely no operating system on it.

That is what you’re leasing that allows you to do a lot of really cool sexy amazing cool stuff um and with the modern technology like I say it’s actually very very very doable today it’s one of those things when you got to start thinking we think about the cloud I mean that’s a whole one of the problems with us old technicians right is we’re used to touching stuff we like you know when we work on computers we’re used to keyboards we’re used to like plugging away and we’ve got the server in front of us and we got the router in front of us we got all this stuff in front of us so like mentally we think about all this equipment like being in our server room being in our office being in our facility and so what you have to realize is in this modern world that we’re in you can have the exact same functionality that you would have if the equipment was in your building but it can be somewhere else it can be provisioned given to you very quickly it can be given to you very inexpensively and you can be provided as securely or more securely than what you could do yourself again a lot of people you know I’m starting to talk you know talking about things like metal as a service and everybody gets worried about security everybody’s like oh how do I how do I know Thanh.

How do I know my servers are going to be secure how do I know that data center is going to be secure well one you do something called due diligence you you actually make sure that the company that you’re dealing with is a legitimate company you probably if you’re going to be running your business office stuff you should fly out to their data center at least once to actually take a look at it make sure they’ve got all the security stuff and all that but beyond that what a lot of people don’t realize is how in secure their facilities actually are they always worry about how insecure the cloud provider is and they somehow completely ignore just the crappy crappy crappy crappy crappy security that they have on their facility again I’m here in Baltimore Maryland in the Baltimore City.

We have an incredibly high crime rate and so one of the real problems that you have is you can have antivirus on your servers you can have your firewalls on your servers you can of your intrusion detection on the servers you can have your ups on the servers and some crackhead could break into your building literally rip the server out of server rack and walk away with it and try to sell for 25 bucks to the local pawn shop and when they can’t sell it at the local pawn shop then they’ll get pissed and they’ll just throw it in a ditch and keep walking again that’s the nice thing with these data centers at least with that kind of physical security you would be surprised many times they have much much much much better physical security in the rest of this then then you have take a real honest hard look at the security in your facility and if you’re honest about it you probably know that it’s it’s probably pretty bad it’s probably probably probably your servers would be better off in some kind of hosted solution so that’s all there is for from metal as a service.

I enjoy taking this class and look forward to see you the next one you.

Python Examples

Basic Programs:

  1. Python program to add two numbers
  2. Maximum of two numbers in Python
  3. Python Program for factorial of a number
  4. Python Program for simple interest
  5. Python Program for compound interest
  6. Python Program to check Armstrong Number
  7. Python Program for Program to find area of a circle
  8. Python program to print all Prime numbers in an Interval
  9. Python program to check whether a number is Prime or not
  10. Python Program for n-th Fibonacci number
  11. Python Program for How to check if a given number is Fibonacci number?
  12. Python Program for n\’th multiple of a number in Fibonacci Series
  13. Program to print ASCII Value of a character
  14. Python Program for Sum of squares of first n natural numbers
  15. Python Program for cube sum of first n natural numbers

Array Programs:

  1. Python Program to find sum of array
  2. Python Program to find largest element in an array
  3. Python Program for array rotation
  4. Python Program for Reversal algorithm for array rotation
  5. Python Program to Split the array and add the first part to the end
  6. Python Program for Find reminder of array multiplication divided by n
  7. Python Program to check if given array is Monotonic

List Programs:

  1. Python program to interchange first and last elements in a list
  2. Python program to swap two elements in a list
  3. Python | Ways to find length of list
  4. Python | Ways to check if element exists in list
  5. Different ways to clear a list in Python
  6. Python | Reversing a List
  7. Python program to find sum of elements in list
  8. Python | Multiply all numbers in the list
  9. Python program to find smallest number in a list
  10. Python program to find largest number in a list
  11. Python program to find second largest number in a list
  12. Python program to find N largest elements from a list
  13. Python program to print even numbers in a list
  14. Python program to print odd numbers in a List
  15. Python program to print all even numbers in a range
  16. Python program to print all odd numbers in a range
  17. Python program to print positive numbers in a list
  18. Python program to print negative numbers in a list
  19. Python program to print all positive numbers in a range
  20. Python program to print all negative numbers in a range
  21. Remove multiple elements from a list in Python
  22. Python – Remove empty List from List
  23. Python | Cloning or Copying a list
  24. Python | Count occurrences of an element in a list
  25. Python | Remove empty tuples from a list
  26. Python | Program to print duplicates from a list of integers
  27. Python program to find Cumulative sum of a list
  28. Python | Sum of number digits in List
  29. Break a list into chunks of size N in Python
  30. Python | Sort the values of first list using second list

Python – INTERVIEW QUESTIONS – 2020 (with answer and algorithm analysis)

Ex1: Merging two sorted list

We have two sorted lists, and we want to write a function to merge the two lists into one sorted list:

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]

Here is our code:

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]
c = []

while a and b:
    if a[0] < b[0]:
        c.append(a.pop(0))
    else:
        c.append(b.pop(0))
        
# either a or b could still be non-empty
print c + a + b

The output:

[1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 18, 19, 21]

A little bit more compact version using list.extend():

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]

a.extend(b)
c = sorted(a)
print c

Note that the list.extend() is different from list.append():

[1, 3, 5, 7, 9, 11, [2, 4, 6]]  # a.append(b)
[1, 3, 5, 7, 9, 11, 2, 4, 6]    # a.extend(b)

Ex2: Get word frequency – initializing dictionary

We’ll see two ways of initializing dictionary by solving word frequency problem.

#!/usr/bin/python
ss = """Nory was a Catholic because her mother was a Catholic, 
and Nory's mother was a Catholic because her father was a Catholic, 
and her father was a Catholic because his mother was a Catholic, 
or had been."""

words = ss.split()
d = {}.fromkeys(words,0)

# or we can use this to initialize a dict
d = {x:0 for x in words}

for w in words:
    d[w] += 1
print d

ds = sorted(d.items(), key=lambda x:x[1], reverse=True)
print(ds)

# or we can use this for the key
import operator
ds2 = sorted(d.items(), key=operator.itemgetter(1), reverse=True)
print(ds2)

We initialized the dictionary with 0 using fromkeys(), and the output should look like this:

{'a': 6, 'Catholic': 3, 'and': 2, 'because': 3, 'her': 3, 'had': 1, 'Nory': 1, 'father': 2, 
"Nory's": 1, 'his': 1, 'been.': 1, 'mother': 3, 'was': 6, 'or': 1, 'Catholic,': 3}

Here is another way of initializing a dictionary:

d = {}
for w in ss.split():
    d[w] = d.get(w,0) + 1
print d

The third way of dictionary initialization using collections.defaultdict(int) which is convenient for counting:

from collections import defaultdict
d = defaultdict(int)
for w in words:
    d[w] += 1
print d

Initializing dictionary with list – I

Sometimes we may want to construct dictionary whose values are lists.

In the following example, we make a dictionary like {‘Country’: [cities,…], }:

cities = {'San Francisco': 'US', 'London':'UK',
        'Manchester':'UK', 'Paris':'France',
        'Los Angeles':'US', 'Seoul':'Korea'}

# => {'US':['San Francisco', 'Los Angeles'], 'UK':[,], ...}

from collections import defaultdict
# using collections.defaultdict()
d1 = defaultdict(list) # initialize dict with list
for k,v in cities.items():
    d1[v].append(k)
print d1

# using dict.setdefault(key, default=None)
d2 = {}
for k,v in cities.items():
       d2.setdefault(v,[]).append(k)
print d2

Output:

defaultdict(<type 'list'>, {'Korea': ['Seoul'], 'US': ['Los Angeles', 'San Francisco'], 'UK': ['Manchester', 'London'], 'France': ['Paris']})
{'Korea': ['Seoul'], 'US': ['Los Angeles', 'San Francisco'], 'UK': ['Manchester', 'London'], 'France': ['Paris']}

Or we can use this code:

#! /usr/bin/python

cities = {'San Francisco': 'US', 'London':'UK',
        'Manchester':'UK', 'Paris':'France',
        'Los Angeles':'US', 'Seoul':'Korea'}

# => {'US':['San Francisco', 'Los Angeles'], 'UK':[,], ...}

a = [v for k,v in cities.items()]
print(a)

ds = {<strong>x:[]</strong> for x in set(a)}
print(ds)

for k,v in cities.items():
	ds[v].append(k)

print(ds)    

Output:

['UK', 'Korea', 'France', 'UK', 'US', 'US']
{'Korea': [], 'UK': [], 'US': [], 'France': []}
{'Korea': ['Seoul'], 'UK': ['Manchester', 'London'], 'US': ['Los Angeles', 'San Francisco'], 'France': ['Paris']}    

Initializing dictionary with list – II

A little bit simpler problem. We have a list of numbers:

L = [1,2,4,8,16,32,64,128,256,512,1024,32768,65536,4294967296]

We want to make a dictionary with the number of digits as the key and list of numbers the value:

 {1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]})

The code looks like this:

L = [1,2,4,8,16,32,64,128,256,512,1024,32768,65536,4294967296]

from collections import defaultdict
d = defaultdict(list)
for i in L:
    d[len(str(i))].append(i)
print d
print {k:v for k,v in d.items()}

Output:

defaultdict(<type 'list'>, {1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]})
{1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]}

More examples: Word Frequency


map, filter, and reduce

Using map, filter, reduce, write a code that create a list of (n)**2 for range(10) for even integers:

l = [x for x in range(10) if x % 2 == 0]
print l

m = filter(lambda x:x % 2 == 0, [x for x in range(10)] )
# m = list( filter(lambda x:x % 2 == 0, [x for x in range(10)] ) ) # python3
print m

o = map(lambda x: x**2, m)
# o = list( map(lambda x: x**2, m) ) # python3
print o


p = reduce(lambda x,y:x+y, o)
# import functools . # python3
# p = functools.reduce(lambda x,y:x+y, o) # python3
print p

Output:

[0, 2, 4, 6, 8]
[0, 2, 4, 6, 8]
[0, 4, 16, 36, 64]
120


Write a function f()

Q: We have the following code with unknown function f(). In f(), we do not want to use return, instead, we may want to use generator.

for x in f(5):
    print x,

The output looks like this:

0 1 8 27 64

Write a function f() so that we can have the output above.

We may use the following f() to get the same output:

def f(n):
   return [x**3 for x in range(5)]

But we want to use generator not using return.

So, the answer should look like this:

def f(n):
    for x in range(n):
        yield x**3

The yield enables a function to comeback where it left off when it is called again. This is the critical difference from a regular function. A regular function cannot comes back where it left off. The yield keyword helps a function to remember its state.

A generator function is a way to create an iterator. A new generator object is created and returned each time we call a generator function. A generator yields the values one at a time, which requires less memory and allows the caller to get started processing the first few values immediately.

Another example of using yield:

Let’s build the primes() function so that I fills the n one at a time, and comes back to primes() function until n > 100.

def isPrime(n):
   if n == 1:
      return False
   for t in range(2,n):
      if n % t == 0:
         return False
   return True

for n in primes():
   print n,

The print out from the for-loop should look like this:

2 3 5 7 11 ... 83 89 97

Here is the primes() function:

def primes(n=1):
   while n < 100:
      # yields n instead of returns n
      if isPrime(n): yield n
      # next call it will increment n by 1
      n += 1


Here is a more practical sample of code which I used for Natural Language Processing(NLP).

Suppose we have a huge data file that has hundred millions of lines. So, it may well exceed our computer’s memory. In this case, we may want to take so called out-of-core approach: we process data in batch (partially, one by one) rather than process it at once. This saves us from the memory issue when we deal with big data set.

So, we want to use yield command. In the following sample, we do process three lines at a time.

Here is the code:

def stream_docs(path):
    with open(path, 'rb') as lines:
        for line in lines:
            text, label = line[:-3], line[-3:-1]
            yield text, label
            
def get_minibatch(doc_stream, size):
    docs, y = [], []
    try:
        for _ in range(size):
            text, label = next(doc_stream)
            docs.append(text)
            y.append(label)
    except StopIteration:
        return None, None
    return docs, y

doc_stream = stream_docs(path='./test.txt')

for _ in range(100):
    X_train, y_train = get_minibatch(doc_stream, size=3)
    if not X_train:
        break
    print 'X_train, y_train=', X_train, y_train

The yield makes the stream_docs() to return a generator which is always an iterator:

generators-iterators-iterables.png

(note) – iterable: strings, lists, tuples, dictionaries, and sets are all iterable objects (containers) which we can get an iterator from. A range() functiuon also returns iterable object. All these objects have an iter() method which is used to get an iterator.

If we comment out the “yield” line, we get “TypeError: NoneType object is not an iterator” at the next(doc_stream) in “get_minibatch()” function.

Output from the code:

X_train, y_train= ['line-a', 'line-b', 'line-c'] [' 1', ' 2', ' 3']
X_train, y_train= ['line-d', 'line-e', 'line-f'] [' 4', ' 5', ' 6']
X_train, y_train= ['line-g', 'line-h', 'line-i'] [' 7', ' 8', ' 9']
X_train, y_train= ['line-j ', 'line-k ', 'line-k '] ['10', '11', '12']

The input used in the code looks like this:

line-a 1
line-b 2
line-c 3
line-d 4
line-e 5
line-f 6
line-g 7
line-h 8
line-i 9
line-j 10
line-k 11
line-k 12

The digit at the end of each line is used as a class label(y_train), so we want to keep it separate from the rest of the text(X_train).


For more information about yield or generator, please visit:

  1. The yield keyword
  2. Generator Functions and Expressions

What is __init__.py?

It is used to import a module in a directory, which is called package import.

If we have a module, dir1/dir2/mod.py, we put __init__.py in each directories so that we can import the mod like this:

import dir1.dir2.mod

The __init__.py is usually an empty py file. The hierarchy gives us a convenient way of organizing the files in a large system.

Build a string with the numbers from 0 to 100, “0123456789101112…”

We may want to use str.join rather than appending a number every time.

>>> ''.join([`x` for x in xrange(101)])
'0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100'
>>> 

Note that the (`) is a backquote not a regiular single quote ():

>>> type(1)
<type 'int'>
>>> type(`1`)
<type 'str'>

Note that we cannot use double quote(“) or single quote(‘) to make n a string:

>>> type("1")
<type 'str'>

>>> ''.join(["n" for n in range(10)])
'nnnnnnnnnn'
>>> ''.join(['n' for n in range(10)])
'nnnnnnnnnn'

>>> n = 1
>>> print `n`
1
>>> print "n"
n

Note: join() returns a string in which the string elements of sequence have been joined by string separator.

We can use str(x) instead:

>>> ''.join([str(x) for x in range(10)])
'0123456789'

Also, since the xrange() is replaced with range in Python 3.x, we should use range() instead for compatibility.

Basic file processing: Printing contents of a file.

try:
    with open('filename','r') as f:
        print f.read()
except IOError:
    print "No such file exists"

How can we get home directory using ‘~’ in Python?

We need to import os module, and add just one line:

import os
print os.path.expanduser('~')

Output:

/home/k

The usage of os.path.dirname() & os.path.basename()

For example, we have the path like this, /home/k/TEST/PYTHON/p.py:

We can get the dir and file using the following:

First, we need to import os module:

>>> import os

Then, we do:

>>> os.path.dirname('/home/k/TEST/PYTHON/p.py')
'/home/k/TEST/PYTHON'

>>> os.path.basename('/home/k/TEST/PYTHON/p.py')
'p.py'

Or we can get them at once in tuple using os.path.split():

>>> os.path.split('/home/k/TEST/PYTHON/p.py')
('/home/k/TEST/PYTHON', 'p.py')

If we want to combine and make a full path:

>>> os.path.join('/home/k/TEST/PYTHON', 'p.py')
'/home/k/TEST/PYTHON/p.py'

Default Libraries

We should be able to answer the questions about the standard library.
Such as “Do you know if there’s a standard library for recursive file renaming?”,
or “In which library would you use for regular expression?”

  1. os: operating system support
    os.path: Common pathname manipulations:
    >>> import os >>> print(os.getcwd()) C:\Python32 >>> cur_dir = os.curdir >>> print(cur_dir) . >>> scripts_dir = os.path.join(os.curdir, ‘Tools\Scripts’) >>> print(scripts_dir) .\Tools\Scripts >>> diff_py = os.path.join(scripts_dir, ‘diff.py’) >>> print(diff_py) .\Tools\Scripts\diff.py >>> os.path.basename(diff_py) ‘ diff.py’ >>> os.path.splitext(diff_py) (‘.\\Tools\\Scripts\\diff’, ‘.py’)
    The os.path.join() function constructs a pathname out of one or more partial pathnames. In this case, it simply concatenates strings.Other convenient ones are: dirname() and basename(), which are the 1st and 2nd element of split(), respectively:>>> import os >>> print(os.getcwd()) C:\TEST\dirA\dirB\dirC >>> print(os.path.dirname(os.getcwd())) C:\TEST\dirA\dirB >>> print(os.path.basename(os.getcwd())) dirC >>> print(os.path.split(os.getcwd())) (‘C:\\TEST\\dirA\\dirB’, ‘dirC’) >>> pathname = os.path.join(os.getcwd(),’myfile.py’) >>> pathname ‘C:\\TEST\\dirA\\dirB\\dirC\\myfile.py’ >>> (dirname, filename) = os.path.split(pathname) >>> dirname ‘C:\\TEST\\dirA\\dirB\\dirC’ >>> filename ‘myfile.py’
    The split function splits a full pathname and returns a tuple containing the path and filename. We could use multi-variable assignment to return multiple values from a function. The os.path.split() function does exactly that. We assign the return value of the split function into a tuple of two variables. Each variable receives the value of the corresponding element of the returned tuple.The first variable, dirname, receives the value of the first element of the tuple returned from the os.path.split() function, the file path. The second variable, filename, receives the value of the second element of the tuple returned from the os.path.split() function, the filename.os.path also contains the os.path.splitext() function, which splits a filename and returns a tuple containing the filename and the file extension.The os.path.expanduser() function :>>> print(os.path.expanduser(‘~’)) C:\Users\KHong
    will expand a pathname that uses ~ to represent the current user’s home directory. This works on any platform where users have a home directory, including Linux, Mac OS X, and Windows. The returned path does not have a trailing slash, but the os.path.join() function doesn’t mind:
  2. re: Regular expression operations
    Visit Regular Expressions with Python
  3. itertools: Functions creating iterators for efficient looping.
    It includes permutations, combinations and other useful iterables.>>> [x for x in itertools.permutations(‘123’)] [(‘1’, ‘2’, ‘3’), (‘1’, ‘3’, ‘2’), (‘2’, ‘1’, ‘3’), (‘2’, ‘3’, ‘1’), (‘3’, ‘1’, ‘2’), (‘3’, ‘2’, ‘1’)] >>> [x for x in itertools.permutations(‘123’,2)] [(‘1’, ‘2’), (‘1’, ‘3’), (‘2’, ‘1’), (‘2’, ‘3’), (‘3’, ‘1’), (‘3’, ‘2’)] >>>

range vs xrange

>>> sum(range(1,101))
5050
>>> sum(xrange(1,101))
5050
>>> 

range() returns a list to the sum function containing all the numbers from 1 to 100. But xrange() returns an iterator rather than a list, which makes it more lighter in terms of memory use as shown below.

>>> range(1,5)
[1, 2, 3, 4]
>>> xrange(1,5)
xrange(1, 5)
>>> 

Note: Since the xrange() is replaced with range in Python 3.x, we should use range() instead for compatibility. The range() in Python 3.x just returns iterator. That means it does not produce the results in memory any more, and if we want to get list from range(), we need to force it to do so: list(range(…)).

Iterators

Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries.

Any object with a __next__() method to advance to a next result is considered iterator. Note that if an object has __iter__() method, we call the object iterable.

iterators-iterables.png

For more info, please visit
http://www.bogotobogo.com/python/python_iterators.php.

Generators

Generators allow us to declare a function that behaves like an iterator, i.e. it can be used in a for loop. It’s a function type generator, but there is another type of generator that may be more familiar to us – expression type generator used in list comprehension:

>>> # List comprehension makes a list
>>> [ x ** 3 for x in range(5)]
[0, 1, 8, 27, 64]
>>> 
>>> # Generator expression makes an iterable
>>> (x ** 3 for x in range(5))
<generator object <genexpr> at 0x000000000315F678>

With the generator expression, we can just wrap it with list() call:

>>> list(x ** 3 for x in range(5))
[0, 1, 8, 27, 64]

Since every generator is an iterator, we can use next() to get the values:

>>> generator = (x ** 3 for x in range(5))
>>> generator.next()
0
>>> generator.next()
1
>>> generator.next()
8
>>> generator.next()
27

For more information about generators, please visit:
http://www.bogotobogo.com/python/python_generators.php

Manipulating functions as first-class objects

Functions as first-class objects?
That means we can pass them around as objects and can manipulate them. In other words, most of the times, this just means we can pass these first-class citizens as arguments to functions, or return them from functions. Everything in Python is a proper object. Even things that are “primitive types” in other languages:

>>> dir (100)
['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', 
'__delattr__', '__div__', '__divmod__', '__doc__', '__float__', 
....
 'numerator', 'real']
>>> 

Functions have attributes and can be referenced and assigned to variables.

>>> def one(arg):
	'''I am a function returning arg I received.'''
	return arg

>>> one(1)
1
>>> one
<function one at 0x0284AA70>
>>> One = one
>>> One
<function one at 0x0284AA70>
>>> one.__doc__
'I am a function returning arg I received.'
>>> 

docstrings vs comments

docstring is the documentation string for a function. We use it as shown below:

function_name.__doc__

We can declare it like this:

def my_function():
    """our docstring"""

or:

def my_function():
    '''our docstring'''

Everything between the triple quotes (with double quotes, “”” or with single quotes,”’) is the function’s docstring, which documents what the function does. A docstring, if it exists, must be the first thing defined in a function. In other words, it should appear on the next line after the function declaration. We don’t technically need to give our function a docstring, but we always should. The docstring will be available at runtime as an attribute of the function.

Writing documentation for our program this way makes the code more readable. We can also use comments for clarification of what the code is doing. In general, docstrings are for documentation, comments are for a code reader.

Monkey-patching

The origin of monkey-patch according to wiki is :
“The term monkey patch seems to have come from an earlier term, guerrilla patch, which referred to changing code sneakily at runtime. The word guerrilla, homophonous with gorilla, became monkey, possibly to make the patch sound less intimidating.”

In Python, the term monkey patch only refers to dynamic modifications of a class or module at runtime, motivated by the intent to patch existing third-party code as a workaround to a bug or feature which does not act as we desire.

We have a module called m.py like this:

# m.py
class MyClass:
    def f(self):
        print "f()"

Then, if we run the monkey-patch testing like this:

>>> import m
>>> def monkey_f(self):
	print "monkey_f()"

	
>>> m.MyClass.f = monkey_f
>>> obj = m.MyClass()
>>> obj.f()
monkey_f()
>>> 

As we can see, we did make some changes in the behavior of f() in MyClass using the function we defined, monkey_f(), outside of the module m.

It is a risky thing to do, but sometimes we need this trick, such as testing.

pdb – The Python Debugger

The module pdb defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame. It also supports post-mortem debugging and can be called under program control.

Using Lambda

Python supports the creation of anonymous functions (i.e. functions that are not bound to a name) at runtime, using a construct called lambda. This is not exactly the same as lambda in functional programming languages such as Lisp, but it is a very powerful concept that’s well integrated into Python and is often used in conjunction with typical functional concepts like filter()map() and reduce().

The following code shows the difference between a normal function definition, func and a lambda function, lamb:

def func(x): return x ** 3

>>> print(func(5))
125
>>> 
>>> lamb = lambda x: x ** 3
>>> print(lamb(5))
125
>>> 

As we can see, func() and lamb() do exactly the same and can be used in the same ways. Note that the lambda definition does not include a return statement — it always contains an expression which is returned. Also note that we can put a lambda definition anywhere a function is expected, and we don’t have to assign it to a variable at all.

The lambda‘s general form is :

lambda arg1, arg2, ...argN : expression using arguments

Function objects returned by running lambda expressions work exactly the same as those created and assigned by defs. However, there are a few differences that make lambda useful in specialized roles:

  1. lambda is an expression, not a statement.
    Because of this, a lambda can appear in places a def is not allowed. For example, places like inside a list literal, or a function call’s arguments. As an expression, lambda returns a value that can optionally be assigned a name. In contrast, the def statement always assigns the new function to the name in the header, instead of returning is as a result.
  2. lambda’s body is a single expression, not a block of statements.
    The lambda‘s body is similar to what we’d put in a def body’s return statement. We simply type the result as an expression instead of explicitly returning it. Because it is limited to an expression, a lambda is less general that a def. We can only squeeze design, to limit program nesting. lambda is designed for coding simple functions, and def handles larger tasks.
>>> 
>>> def f(x, y, z): return x + y + z

>>> f(2, 30, 400)
432

We can achieve the same effect with lambda expression by explicitly assigning its result to a name through which we can call the function later:

>>> 
>>> f = lambda x, y, z: x + y + z
>>> f(2, 30, 400)
432
>>> 

Here, the function object the lambda expression creates is assigned to f. This is how def works, too. But in def, its assignment is an automatic must.

For more, please go to Functions lambda

Properties vs Getters/Setters

We have more detailed discussion in Classes and Instances – Method: Properties.

In general, properties are more flexible than attributes. That’s because we can define functions that describe what is supposed to happen when we need setting, getting or deleting them. If we don’t need this additional flexibility, we may just use attributes since they are easier to declare and faster.

However, when we convert an attribute into a property, we just define some getter and setter that we attach to it, that will hook the data access. Then, we don’t need to rewrite the rest of our code, the way for accessing the data is the same, whatever our attribute is a property or not.

classmethod vs staticmethod

From the official Python documentation on @classmethod:

classmethod(function)
    Return a class method for function.

    A class method receives the class as implicit first argument, 
    just like an instance method receives the instance. 
    To declare a class method, use this idiom:

class C:
    @classmethod
    def f(cls, arg1, arg2, ...): ...

The @classmethod form is a function decorator.
It can be called either on the class (such as C.f()) or on an instance (such as C().f()). 
The instance is ignored except for its class. If a class method is called for a derived class, 
the derived class object is passed as the implied first argument.

And for the @staticmethodthe python doc describes it as below:

staticmethod(function)
    Return a static method for function.

    A static method does not receive an implicit first argument. 
    To declare a static method, use this idiom:

class C:
    @staticmethod
    def f(arg1, arg2, ...): ...

The @staticmethod form is a function decorator.
It can be called either on the class (such as C.f()) or on an instance (such as C().f()). 
The instance is ignored except for its class.
Static methods in Python are similar to those found in Java or C++.

For more info on static vs class methods, please visit:

@static method vs class method

Making a list with unique element from a list with duplicate elements

Iterating the list is not a desirable solution. The right answer should look like this:

>>> dup_list = [1,2,3,4,4,4,5,1,2,7,8,8,10]
>>> unique_list = list(set(dup_list))
>>> print unique_list
[1, 2, 3, 4, 5, 7, 8, 10]
>>> 

Name the functional approach that Python is taking.

Python provides the following:

  1. map(aFunction, aSequence)
  2. filter(aFunction, aSequence)
  3. reduce(aFunction, aSequence)
  4. lambda
  5. list comprehension

These functions are all convenience features in that they can be written in Python fairly easily. Functional programming is all about expressions. We may say that the Functional programming is an expression oriented programming.

What is map?

The syntax of map is:

map(aFunction, aSequence)

The first argument is a function to be executed for all the elements of the iterable given as the second argument. If the function given takes in more than 1 arguments, then many iterables are given.

>>> def cubic(x):
	return x*x*x

>>> items = [x for x in range(11) if x % 2 == 0]
>>> list(map(cubic, items))
[0, 8, 64, 216, 512, 1000]
>>>
>>> list(map(lambda x,y: x*y, [1,2,3], [4,5,6]))
[4, 10, 18]
>>> 

map is similar to list comprehension but is more limited because it requires a function instead of an arbitrary expression.

What is filter and reduce?

Just for comparison purpose, in the following example, we will include map as well.

>>> integers = [ x for x in range(11)]
>>> filter(lambda x: x % 2 == 0, integers)
[0, 2, 4, 6, 8, 10]
>>> map(lambda x: x**2, integers)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
>>> reduce(lambda x, y: x + y, integers)
55
>>> 

In the above example, we defined a simple list of integer values, then we use the standard functions filter(), map() and reduce() to do various things with that list. All of the three functions expect two arguments: A function and a list.
In the first example, filter() calls our lambda function for each element of the list, and returns a new list that contains only those elements for which the function returned “True”. In this case, we get a list of all even numbers.
In the second example, map() is used to convert our list. The given function is called for every element in the original list, and a new list is created which contains the return values from our lambda function. In this case, it computes x^2 for every element.
Finally, reduce() is somewhat special. The function for this one must accept two arguments (x and y), not just one. The function is called with the first two elements from the list, then with the result of that call and the third element, and so on, until all of the list elements have been handled. This means that our function is called n-1 times if the list contains n elements. The return value of the last call is the result of the reduce() construct. In the above example, it simply adds the arguments, so we get the sum of all elements.

*args and **kwargs

Putting *args and/or **kwargs as the last items in our function definition’s argument list allows that function to accept an arbitrary number of anonymous and/or keyword arguments.
Those arguments are called Keyword Arguments. Actually, they are place holders for multiple arguments, and they are useful especially when we need to pass a different number of arguments each time we call the function.

We may want to use *args when we’re not sure how many arguments might be passed to our function, i.e. it allows us to pass an arbitrary number of arguments to your function as shown in the example below:

Let’s make a function that sums of all numbers. It should work for bo the inputs: 1,2,3,4,5 as separate args and as a list [1,2,3,4,5]:

def fn(<strong>*</strong>args):
	ans = 0 
	for x in args:
		ans += x
	return ans

print(fn(1,2,3,4,5))
print(fn(<strong>*</strong>[1, 2, 3, 4, 5]))
print(fn(<strong>*</strong>range(1,6)))

The keyword arguments is a special name=value syntax in function calls that specifies passing by name. It is often used to provide configuration options.

>>> def kargs_function(<strong>**</strong>kargs):
	for k,v in kargs.items():
		print (k,v)

>>> kargs_function(<strong>**</strong>{'uno':'one','dos':'two','tres':'three'})
('dos', 'two')
('tres', 'three')
('uno', 'one')
>>>
>>> kargs_function(dos='two', tres='three', uno='one')
('dos', 'two')
('tres', 'three')
('uno', 'one')
>>> 

For more details, please visit *args and **kwargs – Collecting and Unpacking Arguments.

mutable vs immutable

The content of objects of immutable types cannot be changed after they are created.

immutablemutable
tuple, frozen set, int, float, strlist, set, dict, byte array

Difference between remove, del and pop on lists

To remove a list element, we can use either the del statement if we know exactly which element(s) we are deleting or the remove() method if we do not know.

list.remove(element), del list(index), list.pop(index)

remove() removes the first matching value, not a specific index:

>>> a = [5,6,7,7,8]
>>> a.remove(7)
>>> a
[5, 6, 7, 8]

Both del and pop work on index:

>>> a = [5,6,7,7,8]
>>> del a[1]
>>> a
[5, 7, 7, 8]

>>> a = [5,6,7,7,8]
>>> a.pop(1)
6
>>> a
[5, 7, 7, 8]

>>> a = [5,6,7,7,8]
>>> a.pop(a.index(6)) # get the index for 6
6
>>> a
[5, 7, 7, 8]

Join with new line

Given a list of string, [‘Black’, ‘holes’, ‘are’, ‘where’, ‘God’, ‘divided’, ‘by’, ‘zero’], print each word in a new line:

>>> s = ['Black', 'holes', 'are', 'where', 'God', 'divided', 'by', 'zero']
>>> print '\n'.join(s)
Black
holes
are
where
God
divided
by
zero

Python Tkinter

Graphical User Interface(GUI) is a form of user interface which allows users to interact with computers through visual indicators using items such as icons, menus, windows, etc. It has advantages over the Command Line Interface(CLI) where users interact with computers by writing commands using keyboard only and whose usage is more difficult than GUI.

What is Tkinter?

Tkinter is the inbuilt python module that is used to create GUI applications. It is one of the most commonly used modules for creating GUI applications in Python as it is simple and easy to work with. You don’t need to worry about the installation of the Tkinter module separately as it comes with Python already. It gives an object-oriented interface to the Tk GUI toolkit.

Some other Python Libraries available for creating our own GUI applications areKivyPython QtwxPython

Among all Tkinter is most widely used

What are Widgets?

Widgets in Tkinter are the elements of GUI application which provides various controls (such as Labels, Buttons, ComboBoxes, CheckBoxes, MenuBars, RadioButtons and many more) to users to interact with the application.

Fundamental structure of tkinter program



Basic Tkinter Widgets:

WIDGETSDESCRIPTION
LabelIt is used to display text or image on the screen
ButtonIt is used to add buttons to your application
CanvasIt is used to draw pictures and others layouts like texts, graphics etc.
ComboBoxIt contains a down arrow to select from list of available options
CheckButtonIt displays a number of options to the user as toggle buttons from which user can select any number of options.
RadiButtonIt is used to implement one-of-many selection as it allows only one option to be selected
EntryIt is used to input single line text entry from user
FrameIt is used as container to hold and organize the widgets
MessageIt works same as that of label and refers to multi-line and non-editable text
ScaleIt is used to provide a graphical slider which allows to select any value from that scale
ScrollbarIt is used to scroll down the contents. It provides a slide controller.
SpinBoxIt is allows user to select from given set of values
TextIt allows user to edit multiline text and format the way it has to be displayed
MenuIt is used to create all kinds of menu used by an application

From Thanh : Software Developer | Devops Engineer | Blog Writer in Germany

Simplify Tasks

Want to learn the simple way?

Chuyện của Nam

Nơi những điều nhỏ bé được khơi dậy

ACE IT SKILLS

From Thanh : Volkswagen Software Developer in Germany

In Dianes Kitchen

Recipes showing step by step directions with pictures and a printable recipe card.

ACE IT SKILLS

From Thanh : Software Developer | Devops Engineer | Blog Writer in Germany

The Daily Post

The Art and Craft of Blogging

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.