Saturday, November 25, 2017

Elastic Search - Clustering

In the previous post, we saw how to install and run an elasticsearch instance. In this post, we will see how to cluster it. Clustering is required to support high availability and we will be seeing how to cluster 3 nodes of elasticsearch.

  1. Follow my post to install and run an elastic search node. Repeat it to create multiple nodes on different machines / VM in the same network or multiple nodes on the same machine running on different ports. Running multiple nodes on the same machine comes in handy while developing and researching and can be done by changing the http.port value as described in the next step

  2. Navigate to the config folder inside elasticsearch install location of each of the nodes ie: C:\Deploy\elasticsearch-1.7.2\config. Open the elasticsearch.yml file and update the following settings
    • cluster.name:  clusterXXXPool  -- Name of the cluster to be same across all nodes
    • node.name: "NodeXXX" -- Individual node name
    • bootstrap.mlockall: true -- To lock the memory
    • http.port: 9200 -- Port on which elasticsearch will run
    • discovery.zen.minimum_master_nodes: 2 -- Creates a cluster with minimum 2 master nodes
    • discovery.zen.ping.timeout: 10s
    • discovery.zen.ping.unicast.hosts:["Node1_IP:Port",”Node2_IP:Port","Node3_IP:Port"] -- Lists the node names in cluster

    These are the few settings that can be applied. The config file itself has comments for each setting and provides a good self-documentation if you want to learn more about it.

  3. Restart all the elasticsearch services that are part of the cluster for the settings to be applied.

  4. Use the following URLs on the browser to check cluster related settings
    • NodeIP:Port/_cluster/health – Cluster Health
    • NodeIP:Port/_cat/nodes – Nodes in the cluster
    • NodeIP:Port/_cat/master – Master in the cluster
    • NodeIP:Port/_cluster/settings – Settings of the cluster
    • NodeIP:Port/_cluster/nodes/stats – Statistics of each Node

    To verify if the cluster was set up properly and check its health and status, try to bring down one of the nodes by stopping the service and seeing the output of the above commands.

  5. You can also install Head Plugin to visualize the Cluster Health, Indices and Shards.



Sunday, October 29, 2017

Elasticsearch

Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected. This is a snippet about Elasticsearch from their site https://www.elastic.co/products/elasticsearch. It is a highly efficient index store which offers a great performance.


I have used Elasticsearch for two use cases ie: searching and logging. This blog post will guide you through installing Elasticsearch on a windows machine. As an extension to this post, I will write about clustering and other use cases like ELK in future posts.

Installation
  1. Elasticsearch requires Java Runtime Environment. So go ahead and install it on your Windows Machine.

  2. After installing JRE, navigate to Control Panel --> System and Security --> Advanced System Properties --> Advanced Tab --> Environment Variables and create a new System Variable JAVA_HOME and set and the value to the JRE path.


  3. Download Elasticsearch from https://www.elastic.co/downloads/elasticsearch and extract it to a folder say C:/Deploy/ElasticSearchvx.x.

  4. Open a command prompt in admin mode and change directory to the above folder and run the command service install.This installs Elasticsearch as a service. You can also run it by executing the command elasticsearch.bat 






  5. Open Services window by entering services.msc in Run (Windows + R) and ensure that Elasticsearch service is running.




  6. Navigate to localhost:9200 on your browser and you must see an Elasticsearch status page if everything went well.





  7. You can install Sense Extension for Chrome to interact with elasticsearch
This concludes the simple guide to install Elasticsearch on a Windows Machine as a service. Please share your comments and queries and stay tuned for more posts on this series.

Monday, June 14, 2010

I am DIGIT IDOL – Wow !!!

Wow I couldn’t believe myself when I came to know that I have been made DIGIT IDOL!!! Many of you must be thinking what it is. No it’s not something similar to Indian Idol that you watch on your television. DIGIT is a Tech Magazine and is the India’s most read magazine.

There were around 400 of us who had participated in this contest. I was one among the five who have won the coveted title of Digit Idol. It’s great to see my photo and short snippet about me featured in India’s most read Tech Magazine.

I have been their loyal subscriber right from their birth i.e.: their first issue to till date. I have still preserved all the magazines and it is no less than a treasure for me.

It has been of very great value for the quizzes that I participated. I was two times state finalist of TCS Rural IT Quiz (You could read about it in the older articles on my blog). It helped me identify my interest and truly has navigated me in this Technology world. 

I would go on further recommending everyone to buy and read Digit magazine if possible.

Thank You Digit!!!!!

Thursday, March 25, 2010

TCS - TECH BYTES IT QUIZ - Hurray !!!

TCS TECH BYTES - IT QUIZ

The Board of IT Education Standards (BITES) and Tata Consultancy Services (TCS) Asia’s largest IT company are organising the first edition of the Engineering IT Quiz – “TCS TechBytes”.

The program is an initiative to help students understand technology better - in the process help them enhance their lateral thinking process and inculcate the spirit to compete and excel.
The quiz would serve as an alternative platform to enhance awareness and confidence levels of the aspiring engineers while bringing them in sync with the current developments in the technology domain through an interactive and engaging quiz platform. “TCS TechBytes” strives to provide a better understanding of IT as technology today is an enabler in diverse fields – be it engineering, medicine, law, sports, science, movie, music, government, automobile and telecommunication.

The quiz is open to engineering students of all streams and will be held across Karnataka. It will be hosted by noted Quizmaster ‘Pickbrain’ Giri Balasubramaniam of Greycaps, who also conducts TCS IT Wiz, Rural IT Quiz and Tata Crucible among others.

Detailed information on the quiz, rules and entry forms are listed below

Each institution can send 10 teams (2 students per team). A model questions format has been sent to all engineering institutions to select the 10 Best Teams from their college. The entries of the selected students should be sent before 25th March to bitesitquiz@gmail.com or faxed to 22207270/1 or courier to the following address

THE COORDINATOR
BITES-TCS IT Quiz
BITES
#9, Ground Floor, UNI Building, Thimmaiah Road
Millers Tank Bed, Bangalore – 560 052

Institutions are urged to nominate a coordinator from their institution and provide his / her contact details (email, mobile, phone nos.) to coordinate with BITES.

ABOUT THE QUIZ

Round 1: College Level ·

The College will select 10 Best Teams (2 members per team) by using the model question paper sent to all institutions. The test should be written as a team (2 students per team). The test should not be answered individually. · The college has to send the enclosed entry form of the selected 10 teams before 25th March, 2010 to: The Coordinator, BITES TCS IT Quiz, BITES, # 9 Ground Floor, UNI Building, Thimmaiah Rd, Millers Tank Bed, Bangalore 560 052. Alternately you can email it to bitesitquiz@gmail.com or fax us to 22207270/1.

Round 2: Regional Finals
The Regional Level Quiz will be conducted by the Quizmaster at the following places:
  • Gulbarga – 12th April
    Venue: Appa Institute of Engineering & Technology, Sharana Basaveshwar,
    Institune Complex, Brahmapur, GULBARGA-585 103 
  • Hubli – 15th April
    Venue: B V B College of Engg. & Technology, Vidyanagar, HUBLI-580 031 
  • Tumkur – 17th April
    Venue: Siddganga Institute of Technology, Tumkur-572 103 
  • Mysore – 26th April
    Venue: Sri Jayachamarajendra College of Engg, JSS Technical Institute
    Campus, Mysore-570 006 
  • Mangalore – 28th April
    Venue: Srinivas Institute of Technology, Merlapadavu, Valachil, Farangipete
    Post, Mangalore – 574 143 
  • Bangalore – 30th April
    Venue: RV College of Engineering, R V Vidyanikethan Post, Mysore Road,
     Bangalore - 560 059


 

 Round 3: State Finals
The 6 Regional Winners from Gulbarga, Hubli/Belgaum, Tumkur, Bangalore, Mysore and Mangalore will be invited to participate in the State Finals at Bangalore on 8th May at PES Institute of Technology, 100 Feet Ring Road, BSK III Stage, Bangalore-560085.
The winning team will emerge as the Karnataka State ‘TCS TechBytes’ Champion.

ENTRY AND ELIGIBILITY
There is NO entry or registration fee.
Students from all engineering streams in Karnataka are eligible to participate.
Only 10 teams per college (2 members per team) will be allowed to participate.
Completed Entry Forms for Round 2 should be sent to BITES before 25th March.
Therefore rush the selected teams from your college immediately.

For more details download these PDF files.

Friday, December 11, 2009

My Quizzing Journey

My journey into the computer world started when I was in 3rd standard. It was then that my cousin brother started a Computer Training Centre by name “Abacus Technologies” and I was the first one to get enrolled. In 3rd Std I learnt basics like office suite and operating systems and in 4th I learned C language on a Linux system. That laid down the foundation for everything else to come and I became very interested in them. I still remember when I was in 3rd once I went to the class at 6 in the morning without telling anyone to find the class still shut. I got good scolding from my father. So I began asking my father to buy me a computer. Way back then I had no idea what a computer would cost and I refused to celebrate my birthdays so as to save money for it.

From then on it was no turning back for me. Now I can boast of having learnt Java, Visual Basic, HTML, a bit of everything to be honest and to have used and tested new products and services. Now many take my advice when it comes to buying something related to computers and electronic gadgets. I was also behind the very successful Harry Potter Drama where I did a good use of sound and video effects with the projector.

It was in 9th that I was introduced to the TCS Rural IT Quiz. It had started in 2000, exclusively for rural schools across Karnataka. Tata Consultancy Services in association with the Department of IT, Govt. of Karnataka, came together to plan, what would be India's first Rural IT quiz and a fantastic rural education initiative. Back then I had quite a lame knowledge for an IT Quiz but enough to get past the school and Taluk level. But wasn't successful in further rounds. Even in 10th the same thing happened but then I got the hang of it. I knew what type of questions would come and with my partner Surya we began preparing for it. In 1st PUC we won the regional level and went on to win 2nd at the state level. We were given prizes by the primary and higher education ministers Mr. Hegde and Mr. Limbavali respectively. We won Nokia 3110c and Rs 26,000 each on a whole. More than the prize, the recognition and appreciation we got for our efforts was heartening. And it also inspired others to take part in it the next year.

This year it was the 10th anniversary of the Quiz and hence a special invitation Quiz was to be held between the winners of TCS Rural IT Quiz and the winners of the TCS IT Wiz which is held for the students in urban areas throughout the country. And this year there was something special in stock for me. The last years win had made some very eager to get past us.

I was shocked and at the same time surprised to see some scoring higher marks than us. At first I thought it was a genuine effort but came to know that he had got hold of the leaked question papers of the school round. Every good story needs a villain, in fact it’s quite funny to know that he had been my villain right from 1st std. Working tirelessly towards teasing me, fighting with me, forming sinister plans behind me and from stealing my favorite rubber to even something I equally liked. I am in a way thankful to him because I believe characters like him make me stronger both mentally and socially. The same thing happened here.  Even while our 1st priority was II PUC exams we put in even more effort and were successful in getting selected to the state finals for two consecutive years.

 The finals of the quiz are held as a part of the annual Bangloreit.biz event. This time the final was held at an auditorium in Bangalore instead of the Palace Ground thanks to the recent recession. We were again a bit unlucky and missed the 1st place by a whisker and had to be content with the 3rd place. After the state finals the national finals started and I was blown at the quizzing standards of the national level participants. Venkatesh Prasad had come there to give away prizes and I was blown away by the modesty and simplicity he had come over there. This time we won Rs. 8500 and a Nokia E63 each. In a way I guess it was good that I did not come first as there was a laptop to be grabbed and I would have grown even more addicted to computers on getting the portable version of it. But seriously I was very sad at that moment and would have cried if my parents had not consoled me.

Nevertheless the immense knowledge and interesting facts that I gathered, and the friends I made throughout this journey is literally priceless. Can you believe that the name of HP, by the founders, Bill Hewlett and David Packard, was decided upon tossing a coin! Apple's CEO Steve Jobs takes an annual salary of just $1! When Bill Gates was at school he hacked and programmed the school computer so as to put all the girls in his class; and many more such interesting facts that can inspire anyone to contribute to this huge field.

And finally all this success wouldn't have come to me at the first place if it was not for the support of my parents who in my childhood had not fulfilled my dream of having a computer at home and also for the constant support in any of my endeavors that I pursued. I am equally thankful to our Computer teacher, Vandana Miss who is just short of being a computer herself. And all the Teachers and the Principal of this institution who had all lend their support in their own respective ways.

Hope you enjoyed my journey.

Saturday, July 04, 2009

Recession a Boon!!!!!

Recession brings with it many problems. I need not tell you that. But it also brings with it some positive things. The truth is that recession is not a good thing for anyone. But we will be making a gross mistake if we allow ourselves to think that there are no advantages that come with it. As they say in every dark cloud there is a silver lining. What we should, therefore, do is to look for the silver lining. During the recession efficiency becomes the catchword. There is nothing more efficient than creating an advantage.

According to me Recession is a boon in disguise. I would like to justify my perspective here.

Recession helps save the environment. As the demand for goods decrease, their production too decreases and with it the associated hazardous effects to the environment too decreases. And also helps us conserve our natural resources as less electricity and fuel are used.

Recession gives rise to people oriented policies. We are now seeing most governments adopting citizen focused approach in their economic and developmental policies. These periods makes them think about the people of the country so as to garner support to win elections for the next term.

During this period we are forced to spend money wisely. People are trying to be very thrifty in their spending because they are not sure if they will be having the same income. Moreover it has made people to shop in local stores rather than those big bazaar, etc. We learn to shop for necessity than for luxury.

This period also makes us inculcate healthier habits. That is healthier home cooked food instead of visiting pizza hut or McDonald, which in turn strengthens family ties and also gives a chance to communicate better with family members or that odd game of chess.

Then it helps curb domination by a single company. The recession hits the big companies the hardest. So it gives fair chance for the small companies to compete with their bigger counterparts. More competition means better products. Moreover in this period companies heavily spend on Research and development. So as the recession ends you will be seeing a flurry of technologically advanced products.

It also gives time to rethink about your careers. If you lose your job, don't be sad. May be the job didn't suit you. Moreover in this period many go for higher education to improve their qualifications and hone their skills for a better job after the economy becomes stable.

This is also a good time to start your own business. When the economy is good, there is so much competition around that your business may not even get noticed. A period of recession offers that ideal relaxed ambiance when you can start a small business and work your way to the top. Start-up costs are much lower in a recession than in boom periods. You won't believe me if I tell you that HP (Hewlett Packard) was started during the Great Depression by Bill Hewlett and Dave Packard and today they are multimillionaires. Many such people became millionaires during The Great Depression. Don't miss your chance of becoming one.

You also get very low interest rates in this period. During recession, central banks tend to lower nominal interest rates in order to spur investments in machinery, land, factories, etc. So financing your business could be much cheaper.

As the economy is deflating, prices of goods too tend to fall. You also get great deals on various services in this period. Retailers offer steep discounts. You could go for your dream vacation at lesser expense. Crude oil prices have decreased greatly. Even your cellular companies offer good economic plans during this period. Even home prices fall during this period.

I'm not trying to make light of hardship in any way. Certainly recession sucks big time. All of us have been affected one way or another. If I had the choice, I'd prefer to earn a penny than to lose my job.

The problem is thats not how it's working at the moment. We're running into the reality that the world as we know it has changed, and like it or not, we must change with it.

I hope my views will help bring new ideas about recession and help you to look at the positive possibilities it brings along. And moreover imagine the world beyond the current recession.

Wednesday, April 15, 2009

Global Warming – What can we do to Protect and Save our Earth

Earth has warmed by about 1ºF over the past 100 years. But why? And how? Well, scientists are not exactly sure. The Earth could be getting warmer on its own, but many of the world's leading climate scientists think that things people do are helping to make the Earth warmer.
Global warming refers to an average increase in the Earth's temperature, which in turn causes changes in climate. A warmer Earth may lead to changes in rainfall patterns, a rise in sea level, and a wide range of impacts on plants, wildlife, and humans.

The greenhouse effect is the rise in temperature that the Earth experiences because certain gases in the atmosphere (water vapour, carbon dioxide, nitrous oxide, and methane, for example) trap energy from the sun. Without these gases, heat would escape back into space and Earth’s average temperature would be about 60ºF colder. Because of how they warm our world, these gases are referred to as greenhouse gases.
Earth's climate has been changing constantly over its 5-billion-year history. Sometimes, the climate has warmed so that the oceans have risen and covered much of the Earth and sometimes so cold that most parts was covered with sheets of ice. Each of the changes may seem extreme, but they usually occurred slowly over many thousands of years.

Scientists are not fortune-tellers. They don't know exactly what will happen in the future. Together, the melting glaciers, rising seas, and computer models provide some good clues. They tell us that the Earth's temperature will probably continue to rise as long as we continue increasing the amount of greenhouse gases in the atmosphere.
It may seem hard to believe that people can actually change the Earth’s climate. Once, all climate changes occurred naturally. However, during the Industrial Revolution, we began altering our climate and environment through agricultural and industrial practices. The Industrial Revolution was a time when people began using machines to make life easier. It started more than 200 years ago and changed the way humans live. Before the Industrial Revolution, human activity released very few gases into the atmosphere, but now through population growth, fossil fuel burning, and deforestation, we are affecting the mixture of gases in the atmosphere.

Since the Industrial Revolution, the need for energy to run machines has steadily increased. Some energy, like the energy you need to do your homework, comes from the food you eat. But other energy, like the energy that makes cars run and much of the energy used to light and heat our homes, comes from fuels like coal and oil – fossil fuels. Burning these fuels releases greenhouse gases.

When Do You Send Greenhouse Gases into the Air?
Whenever you ...
• Watch TV
• Use the Air Conditioner
• Turn on a Light
• Use a Hair Dryer
• Ride in a Car
• Play a Video Game
• Listen to a Stereo
• Use a Dish Washer
• Microwave a Meal
... you are helping to send greenhouse gas into the air. To perform many of these functions, you need to use electricity. Electricity comes from power plants. Most power plants use coal and oil to make electricity. Burning coal and oil produces greenhouse gases.
Other things we do send greenhouse gases into the air too. The trash that we send to landfills produces a greenhouse gas called methane. Methane is also produced by the animals we raise for dairy and meat products and when we take coal out of the ground. Whenever we drive or ride in a car, we are adding greenhouse gases to the atmosphere. And, when factories make the things that we buy and use everyday, they too are sending greenhouse gases into the air.
Sometimes little things can turn into big things. Think about brushing your teeth. If you don't brush for one day, chances are nothing bad will happen. But if you don't brush your teeth for one month, you may develop a cavity. It's the same thing with global temperatures. If temperatures rise above normal levels for a few days, it's no big deal – the Earth will stay more or less the same. But if temperatures continue to rise over a longer period of time, then the Earth may experience some problems.

Climate change may affect people's health both directly and indirectly. For example, heat stress and other heat related health problems are caused directly by very warm temperatures and high humidity. Indirectly, ecological disturbances, air pollution, changes in food and water supplies, and coastal flooding are all examples of possible impacts that might affect human health.
Climate change may be a big problem, but there are many little things we can do to make a difference. If we try, most of us can do our part to reduce the amount of greenhouse gases that we put into the atmosphere. Many greenhouse gases come from things we do every day. As we have learned, these greenhouse gases trap energy in the atmosphere and make the Earth warmer.

Driving a car or using electricity is not wrong. We just have to be smart about it. Some people use less energy by carpooling. For example, four people can ride together in one car instead of driving four cars to work. Here are some additional ways you can help make the planet a better place!
• Learning about the environment is very important. There are many good books that will help you learn about the environment and climate changes.

• Whenever we use electricity, we help put greenhouse gases into the air. By turning off lights, the television, and the computer when you are through with them, you can help a lot.
• Wash clothes in warm or cold water and not in hot water.

• Replace your incandescent bulbs with more efficient compact fluorescents.
• You can save energy by sometimes taking the bus, riding a bi-cycle, or walking.

• Planting trees is fun and a great way to reduce greenhouse gases. Trees absorb carbon dioxide, a greenhouse gas, from the air.
• Recycle cans, bottles, plastic bags, and newspapers. When you recycle, you send less trash to the landfill and you help save natural resources, like trees, oil, and elements such as aluminium.

• Did you know that you can help the environment if you buy recyclable products instead of non-recyclable ones? Look for the recycle mark – three arrows that make a circle – on the package. Recyclable products are usually made out of things that already have been used. It usually takes less energy to make recycled products than to make new ones. The less energy we use, the better.
• The sun produces solar energy. Solar energy is a fancy way of saying "energy that comes from the sun". Solar energy can be used to heat homes, buildings, water, and to make electricity.

• Cars are an important part of life for most people. But cars also cause pollution and release a lot of greenhouse gases into the air. When you buy your next car, look for the one with the best fuel economy in its class.
• Many things, like computers, TVs, stereos, and VCRs, have special labels on them. The label says "Energy" and has a picture of a star. Products with the ENERGY STAR® label are made to save energy. Buying products with ENERGY STAR® labels will help protect the environment.

• Share information about recycling and energy conservation with your friends, neighbours and co-workers, and take opportunities to encourage public officials to establish programs and policies that are good for the environment.