[Ubuntu] Install Hadoop 3.0.0 & Hive on Ubuntu 16.04

What’s Hadoop?

Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

The project includes these modules:

Hadoop Common: The common utilities that support the other Hadoop modules.
Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
Hadoop YARN: A framework for job scheduling and cluster resource management.
Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

What’s Hive?

The Hive data warehouse software facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. Structure can be projected onto data already in storage. A command line tool and JDBC driver are provided to connect users to Hive.

閱讀更多[Ubuntu] Install Hadoop 3.0.0 & Hive on Ubuntu 16.04

(Visited 1,216 times, 1 visits today)

[Ubuntu] Install Spark2.2.1 on Ubuntu 16.04

What is Spark?

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

You can download Spark from here, or follow the below instructions to download and install Spark2.2.1.

閱讀更多[Ubuntu] Install Spark2.2.1 on Ubuntu 16.04

(Visited 802 times, 1 visits today)

[WP] How to Add SSL and HTTPS in WordPress?

SSL plays an important role in website, because in the last year Google announced that they will be using HTTPS and SSL as a ranking signal in their search results. This means that using HTTPS and SSL will help improve your site’s SEO.

Using an SSL certificate for your WordPress site means that your data, as well as your users’ data, remains safe from prying eyes. Since WordPress is a particularly large target for hackers due to its popularity and transparency, it’s important to take all the appropriate steps toward securing your site.

If you host a wordpress website by yourself, you may need to install SSL certificate on your website. I found a simple way to install SSL certificate through commands and the most important thing is this SSL certificate is free. Oh yes, you need to have the basic knowledge of SSH, Linux and Bash.

閱讀更多[WP] How to Add SSL and HTTPS in WordPress?

(Visited 102 times, 1 visits today)

[Python] Use python boto3 and crontab to backup MySQL to AWS S3

I wrote a python script to backup MySQL and upload to AWS S3. I will show you these steps in the following. First of all, you need to create a user in AWS IAM.

閱讀更多[Python] Use python boto3 and crontab to backup MySQL to AWS S3

(Visited 836 times, 1 visits today)

[Ubuntu] Using Amazon RDS with WordPress

In order to improve efficiency and elasticity of EC2 instance, we need to use a database that is installed on a separate machine. A good solution is using AWS RDS. Before we create a database RDS instance, we need to make a dump of the existing database on our WordPress server.

In the following, I will list these instructions.

閱讀更多[Ubuntu] Using Amazon RDS with WordPress

(Visited 115 times, 1 visits today)

[Ubuntu] Install WordPress on AWS EC2 Ubuntu 14.04

I change my blog from google blogger to wordpress recently. First of all, I got an Amazon Web Service account, then I created 2 instances(EC2 and RDS), finally, I installed wordpress on EC2. I will list instructions in the following steps.

1. Create AWS EC2 instance
2. Install Linux, Apache, MySQL, PHP (LAMP) stack on Ubuntu 14.04
3. Install WordPress on Ubuntu 14.04
4. Setup FTP server for wordpress
5. (Option) Create AWS RDS instance and connect to WordPress

閱讀更多[Ubuntu] Install WordPress on AWS EC2 Ubuntu 14.04

(Visited 230 times, 1 visits today)