Git Product home page Git Product logo

example.v2's Introduction

example.v2

An example project for book 'Go Programming & Concurrency in Practice, 2nd edition' (《Go并发编程实战》第2版).

example.v2's People

Contributors

abirdcfly avatar errpunk avatar hyper0x avatar jiaxwu avatar linehk avatar rovast avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

example.v2's Issues

关于第五章Cmap的问题

老师您好,在看第五章Cmap实战的时候有一个问题,bucket delete方法用新链表替换旧链表的时候,会出现新链表和旧链表同时存在的状态,请问旧链表的内存回收是否会出现问题?

如果有线程阻塞在旧链表的某处,这时候旧链表某节点被回收肯定会引发错误,请问go的垃圾回收机制如何知道适当的回收时机呢?

chanbase1.go 得到的结果与书上不符合

书上的 chanbase1.go 的运行结果与实际运行结果(win10,go 1.8.1)不同。书上的结果 Sent: d[sender] 在实际运行时会出现在 Received: d之后。我想容易对读者出现困扰。是否可以在示例的
书上的 Sent :d 会在Received 之前出现。与实际运行结果不符合(win10 go 1.8,1)可以考虑在示例中

for {
            if elem, ok := <-strChan; ok {
                fmt.Println("Received:", elem, "[Receiver]")
                time.Sleep(time.Second) //这里
            } else {
                break
            }
        }

多加一个 time.Second 这样的话可以让读者更好地理解

关于第五章Cmap的bucket的Delete方法是否可以简化一下

  • 老师您好,在cmap里面bucketDelete方法是否可以简化一下 .我们可以直接记录删除要删除节点的pre节点和next节点.把pre.next = next节点

  • 书里面的示例代码:

func (b *bucket) Delete(key string, lock sync.Locker) bool {
	if lock != nil {
		lock.Lock()
		defer lock.Unlock()
	}
	firstPair := b.GetFirstPair()
	if firstPair == nil {
		return false
	}
	var prevPairs []Pair
	var target Pair
	var breakpoint Pair
	for v := firstPair; v != nil; v = v.Next() {
		if v.Key() == key {
			target = v
			breakpoint = v.Next()
			break
		}
		prevPairs = append(prevPairs, v)
	}
	if target == nil {
		return false
	}
	newFirstPair := breakpoint
	for i := len(prevPairs) - 1; i >= 0; i-- {
		pairCopy := prevPairs[i].Copy()
		pairCopy.SetNext(newFirstPair)
		newFirstPair = pairCopy
	}
	if newFirstPair != nil {
		b.firstValue.Store(newFirstPair)
	} else {
		b.firstValue.Store(placeholder)
	}
	atomic.AddUint64(&b.size, ^uint64(0))
	return true
}
  • 我实现的代码
func (b *bucket) Delete(key string, locker sync.Locker) bool {
	if locker != nil {
		locker.Lock()
		defer locker.Unlock()
	}

	firstPair := b.GetFirstPair()
	if firstPair == nil {
		return false
	}
	var prePair IPair
	var target IPair
	var nextPair IPair
	for v := firstPair; v != nil; v = v.Next() {
		if v.Key() == key {
			target = v
			nextPair = v.Next()
			break
		}
		prePair = v
	}

	if target == nil {
		return false
	}

	// bucket只存在一个元素
	if prePair == nil {
		b.firstValue.Store(placeholder)
	} else {
		prePair.SetNext(nextPair)
	}

	atomic.AddUint64(&b.size, ^uint64(0))

	return true
}

image

5.3.6 ConcurrentArray的Set方法实现疑问

chapter5/value/cow/cow.goSet方法:

	newArray := make([]int, array.length)
	copy(newArray, array.val.Load().([]int))
	newArray[index] = elem
	array.val.Store(newArray)

这样实现是否会有并发问题?复制和设值这段是否需要加锁?

关于loadgen中并发的计数方式的疑问

书中,loadgen的并发设计是根据过期时间/时间间隔的方式~
想请问:为何不直接使用for循环concurrency次+timeout的形式来处理呢?这样不是免去了tick的使用?

微调代码,以符合 io.Read 接口定义

apipe.go

ref: https://golang.org/pkg/io/#Reader

// Callers should always process the n > 0 bytes returned before
// considering the error err. Doing so correctly handles I/O errors
// that happen after reading some bytes and also both of the
// allowed EOF behaviors.

for {
	tempOutput := make([]byte, 5)
	n, err := stdout0.Read(tempOutput)
	if err != nil {
		if err == io.EOF {
			break
		} else {
			fmt.Printf("Error: Couldn't read data from the pipe: %s\n", err)
			return
		}
	}
	if n > 0 {
		outputBuf0.Write(tempOutput[:n])
	}
}
for {
	tempOutput := make([]byte, 5)
	n, err := stdout0.Read(tempOutput)
	if n > 0 {
		outputBuf0.Write(tempOutput[:n])
	}
	if err != nil {
		if err == io.EOF {
			break
		} else {
			fmt.Printf("Error: Couldn't read data from the pipe: %s\n", err)
			return
		}
	}
}

书中的废话太多

让我想起那个 笑话

四块
十块?
不是十块 而是四块
二十四块?
不是二十四块 就是四块
九十四块?

这几本编程书下来
以后买书会有个标准
大于300页的烂书率高

关于5.7中 findSegment的疑问

cmap.segments = make([]Segment, concurrency);
假如 concurrency为6
那么int(keyHash32 >> 1) % 5的值为 0 到 4
那么有一个 segment 是不是会一直取不到

错别字

  • 3.2.2 关于同步 第一段 只要它们之存在数据共享 应该是 只要它们之存在数据共享

第五章,线程安全的map实现cmap这里,其实是不安全的。

具体部分是bucket 实现这里。
假如不提供锁参数,单靠原子更新表头,会不会无法保证线程安全?

举个例子,bucket.delete这里,您的实现是copy被删除元素的之前元素链,和之后元素,然后重新构建一条新的单向链表,原子更新firstValue,链表头。

问题就出在这里,假如Copy组成新链表的同一时刻,暂未更新链表头。这时候别的线程操作,更新,或者删除这个bucket链表段中的元素,那么当前copy的这段链表段就不是最新的数据了,然后用这条脏的链表段,更新表头,会覆盖了之前的操作,导致之前别的线程操作的丢失。

问题复现:
bucket.go 文件中,L135之前,添加time.Sleep(time.Second) 放大模拟一个删除延时。

bucket_test.go

func TestBucketDelaySecDeleteNoLocker(t *testing.T) {
	b := newBucket()
	wg := sync.WaitGroup{}
	wg.Add(2)
	p, _ := newPair("A", "a")
	
	b.Put(p, nil)
	p, _ = newPair("B", "b")
	b.Put(p, nil)
	
	p, _ = newPair("C", "c")
	b.Put(p, nil)
	
	//c b a
	t.Logf("before bucket:\n%v\n", b)
	
	go func() {
		defer wg.Done()
		p, _ := newPair("D", "d")
		_, err := b.Put(p, nil)
		if err != nil {
			t.Fatalf("An error occurs when putting a pair to the bucket: %s (pair: %#v)", b, p)
		}
		// bucket should be  D C B A
		t.Logf("after put item D,  bucket:\n%v\n", b)
	}()
	
	go func() {
		defer wg.Done()
		// delete item C
		b.Delete("C", nil)
	}()
	wg.Wait()
	expected := []string{"D", "B", "A"}
	actual := make([]string, 0)
	
	for curr := b.GetFirstPair(); curr != nil; curr = curr.Next() {
		actual = append(actual, curr.Key())
	}
	
	t.Logf("after delete C , bucket:\n%v\n", b)
	
	if !reflect.DeepEqual(actual, expected) {
		t.Fatalf("expected %v , but %v", expected, actual)
	}
}
=== RUN   TestBucketDelaySecDeleteNoLocker
    bucket_test.go:323: before bucket:
        [ pair{key:C, hash:67, element:c, nextKey:B} pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
    bucket_test.go:333: after put item D,  bucket:
        [ pair{key:D, hash:68, element:d, nextKey:C} pair{key:C, hash:67, element:c, nextKey:B} pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
    bucket_test.go:349: after delete C , bucket:
        [ pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
    bucket_test.go:352: expected [D B A] , but [B A]
--- FAIL: TestBucketDelaySecDeleteNoLocker (1.01s)
FAIL

socket server端代码疑问

chapter3/socket/tcp_socket.go 中,server 端的 wg.Done() 放在 handleConn() 中,会导致 server 端的 listener.Close() 无法被执行吧?

golang.org/x/net

这个包 import "golang.org/x/net/..." 要到哪里下载啊,我到墙外也没找到下载链接啊

有一些疑问需要解答

package analyzer
....
// New 用于创建一个分析器实例。
func New(
	mid module.MID,
	respParsers []module.ParseResponse,
	scoreCalculator module.CalculateScore) (module.Analyzer, error) {
	moduleBase, err := stub.NewModuleInternal(mid, scoreCalculator)
	if err != nil {
		return nil, err
	}
	if respParsers == nil {
		return nil, genParameterError("nil response parsers")
	}
	if len(respParsers) == 0 {
		return nil, genParameterError("empty response parser list")
	}
	var innerParsers []module.ParseResponse
	for i, parser := range respParsers {
		if parser == nil {
			return nil, genParameterError(fmt.Sprintf("nil response parser[%d]", i))
		}
		innerParsers = append(innerParsers, parser)
	}
	return &myAnalyzer{
		ModuleInternal: moduleBase,
		respParsers:    innerParsers,
	}, nil
}
	var innerParsers []module.ParseResponse
	for i, parser := range respParsers {
		if parser == nil {
			return nil, genParameterError(fmt.Sprintf("nil response parser[%d]", i))
		}
		innerParsers = append(innerParsers, parser)
	}
  • 为什么这里检查respParsers参数,需要把值赋值到一个新的变量innerParsers里面去.不赋值给新的变量我感觉也没什么问题啊!也不存在并发问题.如果参数出错了就直接return了.
  • 并且我在Pipline里面也是看到这样的代码.
  • 希望老师解答一下,要不然我觉得这里肯定有什么阴谋,或者还是老师写代码的习惯.😂😂😂😂😂😂😂😂😂

实测输出结果与《Go并发编程实战》给出的不一致

操作系统:win10,64位
go版本:1.10.3 windows/amd64
代码:书中“1.5 问候程序”->"代码清单 1-2 hello.go"
我运行的代码如下:

package main

import(
	"bufio"
	"fmt"
	"os"
)

func main()  {
	inputReader := bufio.NewReader(os.Stdin)
	fmt.Println("Please input your name:")
	input, err := inputReader.ReadString('\n')
	if err != nil {
		fmt.Printf("An error occurred: %s\n", err)
		os.Exit(1)
	}else {
                input = input[:len(input)-1]
		fmt.Println(input)
		fmt.Printf("Hello, %s!", input)
	}
}

运行结果:

Please input your name:
Robert
Robert
!ello, Robert

以上代码与书中基本一致,书中给出的运行结果为:

Please input your name:
Robert
Hello, Robert!

请问为什么会出现不同的结果呢?

apipe example 运行出错

Run command echo -n "My first command comes from golang.":
My first command comes from golang.

Run command ps aux | grep apipe:
Error: Couldn't wait for the second command: exit status 1

chapter 6的analyzer的Analyze()方法是否会导致内存泄露

  • chapter6/webcrawler/module/local/analyzer/analyzer.go
func (analyzer *myAnalyzer) Analyze(
	resp *module.Response) (dataList []module.Data, errorList []error) {
	analyzer.ModuleInternal.IncrHandlingNumber()
	defer analyzer.ModuleInternal.DecrHandlingNumber()
	analyzer.ModuleInternal.IncrCalledCount()
	if resp == nil {
		errorList = append(errorList,
			genParameterError("nil response"))
		return
	}
	httpResp := resp.HTTPResp()
	if httpResp == nil {
		errorList = append(errorList,
			genParameterError("nil HTTP response"))
		return
	}
	httpReq := httpResp.Request
	if httpReq == nil {
		errorList = append(errorList,
			genParameterError("nil HTTP request"))
		return
	}
	var reqURL = httpReq.URL
	if reqURL == nil {
		errorList = append(errorList,
			genParameterError("nil HTTP request URL"))
		return
	}
	analyzer.ModuleInternal.IncrAcceptedCount()
	respDepth := resp.Depth()
	logger.Infof("Parse the response (URL: %s, depth: %d)... \n",
		reqURL, respDepth)

	// 解析HTTP响应。
	if httpResp.Body != nil {
		defer httpResp.Body.Close()
	}
	multipleReader, err := reader.NewMultipleReader(httpResp.Body)
	if err != nil {
		errorList = append(errorList, genError(err.Error()))
		return
	}
	dataList = []module.Data{}
	for _, respParser := range analyzer.respParsers {
		httpResp.Body = multipleReader.Reader()
		pDataList, pErrorList := respParser(httpResp, respDepth)
		if pDataList != nil {
			for _, pData := range pDataList {
				if pData == nil {
					continue
				}
				dataList = appendDataList(dataList, pData, respDepth)
			}
		}
		if pErrorList != nil {
			for _, pError := range pErrorList {
				if pError == nil {
					continue
				}
				errorList = append(errorList, pError)
			}
		}
	}
	if len(errorList) == 0 {
		analyzer.ModuleInternal.IncrCompletedCount()
	}
	return dataList, errorList
}
	// 解析HTTP响应。
	if httpResp.Body != nil {
		defer httpResp.Body.Close()
	}
	multipleReader, err := reader.NewMultipleReader(httpResp.Body)
	if err != nil {
		errorList = append(errorList, genError(err.Error()))
		return
	}
	dataList = []module.Data{}
	for _, respParser := range analyzer.respParsers {
		httpResp.Body = multipleReader.Reader()
		pDataList, pErrorList := respParser(httpResp, respDepth)
                .....
        }
  • httpResp.Body = multipleReader.Reader() 这里httpResp.Body赋了一个新值,是否会导致原来的Body没有关闭从而造成内存泄漏

为什么会形成竞态条件!

cow.go 41- 44 行
`func (array *intArray) Set(index uint32, elem int) (err error) {
if err = array.checkIndex(index); err != nil {
return
}
if err = array.checkValue(); err != nil {
return
}

// 不要这样做!否则会形成竞态条件!
// oldArray := array.val.Load().([]int)
// oldArray[index] = elem
// array.val.Store(oldArray)

newArray := make([]int, array.length)
copy(newArray, array.val.Load().([]int))
newArray[index] = elem
array.val.Store(newArray)
return

}`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.