Git Product home page Git Product logo

Comments (16)

doddys avatar doddys commented on July 21, 2024

EC2 instance has been created.
Using CentOS 6.9 m4.xlarge

from sebc.

doddys avatar doddys commented on July 21, 2024

Performing System Configuration Checks

from sebc.

doddys avatar doddys commented on July 21, 2024

Submitting Result of System Configuration Checks

  1. Check vm.swappiness on all your nodes
    Set the value to 1 if necessary
  2. Show the mount attributes of your volume(s)
    If you have ext-based volumes, list the reserve space setting
  3. XFS volumes do not support reserve space
  4. Disable transparent hugepage support
  5. List your network interface configuration
  6. Show that forward and reverse host lookups are correctly resolved
  • For /etc/hosts, use getent
  • For DNS, use nslookup
  1. Show the nscd service is running
  2. Show the ntpd service is running

from sebc.

doddys avatar doddys commented on July 21, 2024

Setting up mysql and mysql replication

from sebc.

doddys avatar doddys commented on July 21, 2024

Current issue:
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

from sebc.

doddys avatar doddys commented on July 21, 2024

@manojsundaram could you review my preinstall check?

from sebc.

doddys avatar doddys commented on July 21, 2024

slave replication is still not working. skipping it for now

from sebc.

manojsundaram avatar manojsundaram commented on July 21, 2024

@doddys , please "assign" the issue to me next time you want me to take a look. that gives me a notification

from sebc.

manojsundaram avatar manojsundaram commented on July 21, 2024

Does your my.cnf file look similar to this:
https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5_2

from sebc.

doddys avatar doddys commented on July 21, 2024

Installing Cloudera Manager now

from sebc.

doddys avatar doddys commented on July 21, 2024

@manojsundaram yes i copied from that except this line
binlog_format = mixed
changed to
binlog_format = ROW

from the lab guide

from sebc.

doddys avatar doddys commented on July 21, 2024

Master my.cnf

# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
server-id=1

transaction-isolation = READ-COMMITTED

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M


# For MySQL version 5.1.8 or later. For older versions, reference MySQL documentation for configuration help.
binlog_format = ROW

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M


log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log_bin=/var/lib/mysql/mysql_binary_log
binlog-ignore-db = "mysql"

sql_mode=STRICT_ALL_TABLES

Slave my.cnf

# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M

server-id=2

transaction-isolation = READ-COMMITTED

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M


# For MySQL version 5.1.8 or later. For older versions, reference MySQL documentation for configuration help.
binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M


log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log_bin=/var/lib/mysql/mysql_binary_log
relay-log=/var/lib/mysql/mysql_relay_log

sql_mode=STRICT_ALL_TABLES

from sebc.

doddys avatar doddys commented on July 21, 2024

adjusting heap size based on warning in cloudera manager

from sebc.

doddys avatar doddys commented on July 21, 2024

@manojsundaram should I ignore this warning?

Memory on host hadoop7.expecc.com is overcommitted. The total memory allocation is 13.2 GiB bytes but there are only 15.6 GiB bytes of RAM (3.1 GiB bytes of which are reserved for the system). Visit the Resources tab on the Host page for allocation details. Reconfigure the roles on the host to lower the overall memory allocation. Note: Java maximum heap sizes are multiplied by 1.3 to approximate JVM overhead.

from sebc.

doddys avatar doddys commented on July 21, 2024

I did the bonus step. please review

from sebc.

doddys avatar doddys commented on July 21, 2024

Issue with MySQL Replication is fixed.

The solution was to restore a clean back from master to slave.

from sebc.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.