Istio Prometheus

我选择 Istio 而非 Service Mesh 鼻祖 Linkerd 的原因,是 Istio 有大牌厂商支持、社区生态圈优势、重点支持 Kubernetes。

1   遥测插件的远程访问

这里有很多种方法:

  • 直接用 pod ip + pod port 去访问,
  • 把 pod ip + pod port 映射到 istio 的 virtualservice
  • 把 pod ip + pod port 映射到宿主机的端口

比如这些 Pod 内部开放的端口:Prometheus:9090、Grafana:3000、Kiali:20001、Tracing:80,后两者我们演示一下。

1.1    istio

istio-ingressgateway 服务开放了 15020:31966/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:32293/TCP,15030:30857/TCP,15031:30499/TCP,15032:31537/TCP,15443:32082/TCP 这些端口到宿主机,可以新建 istio 资源进行访问,官方有文档“遥测插件的远程访问

1.2    port-forward 命令转发

比如 Prometheus: kubectl -n istio-system port-forward prometheus-d8d46c5b5-vb87x 9090:9090 & ssh -N -f -L 0.0.0.0:9091:127.0.0.1:9090 yhdodo19@0.0.0.0 -i ~/.ssh/googledodo http://35.237.188.250:9091

Grafana: kubectl -n istio-system port-forward grafana-67c69bb567-qfzm5 3000:3000 & ssh -N -f -L 0.0.0.0:9092:127.0.0.1:3000 yhdodo19@0.0.0.0 -i ~/.ssh/googledodo http://35.237.188.250:9092

kubectl -n istio-system port-forward istio-ingressgateway-5d8d989c76-cctpl 15000:15000 & ssh -N -f -L 0.0.0.0:9093:127.0.0.1:15000 yhdodo19@0.0.0.0 -i ~/.ssh/googledodo http://35.237.188.250:9093/listeners 查看该pod的监听器,默认只有 0.0.0.0:15090,比如我添加了 0.0.0.0:443,就可以接受 istio-ingressgateway svc 的 443 接口 http://35.237.188.250:9093/config_dump

ssh -N -f -L 0.0.0.0:80:127.0.0.1:31390 yhdodo19@0.0.0.0 -i ~/.ssh/googledodo

{
    "configs":[
        {
            "@type":"type.googleapis.com/envoy.admin.v2alpha.BootstrapConfigDump",
            "bootstrap":Object{...},
            "last_updated":"2019-06-25T10:15:56.689Z"
        },
        {
            "@type":"type.googleapis.com/envoy.admin.v2alpha.ClustersConfigDump",
            "version_info":"2019-06-28T03:17:42Z/49",
            "static_clusters":Array[3],
            "dynamic_active_clusters":Array[100]
        },
        {
            "@type":"type.googleapis.com/envoy.admin.v2alpha.ListenersConfigDump",
            "version_info":"2019-06-28T03:17:42Z/49",
            "static_listeners":Array[1],
            "dynamic_active_listeners":Array[1]
        },
        {
            "@type":"type.googleapis.com/envoy.admin.v2alpha.RoutesConfigDump",
            "static_route_configs":Array[1],
            "dynamic_route_configs":[
                {
                    "version_info":"2019-06-28T03:17:42Z/49",
                    "route_config":{
                        "name":"http.443",
                        "virtual_hosts":[
                            {
                                "name":"kube.jemper.cn:443",
                                "domains":[
                                    "kube.jemper.cn",
                                    "kube.jemper.cn:443"
                                ],
                                "routes":[
                                    {
                                        "match":{
                                            "prefix":"/sec"
                                        },
                                        "route":{
                                            "cluster":"outbound|82|v3|goapisec.default.svc.cluster.local",
                                            "timeout":"0s",
                                            "retry_policy":{
                                                "retry_on":"connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
                                                "num_retries":2,
                                                "retry_host_predicate":[
                                                    {
                                                        "name":"envoy.retry_host_predicates.previous_hosts"
                                                    }
                                                ],
                                                "host_selection_retry_max_attempts":"5",
                                                "retriable_status_codes":[
                                                    503
                                                ]
                                            },
                                            "max_grpc_timeout":"0s"
                                        },
                                        "metadata":{
                                            "filter_metadata":{
                                                "istio":{
                                                    "config":"/apis/networking/v1alpha3/namespaces/default/virtual-service/goapi-default-xixi"
                                                }
                                            }
                                        },
                                        "decorator":{
                                            "operation":"goapisec.default.svc.cluster.local:82/sec*"
                                        },
                                        "per_filter_config":{
                                            "mixer":{
                                                "disable_check_calls":true,
                                                "forward_attributes":{
                                                    "attributes":{
                                                        "destination.service.host":{
                                                            "string_value":"goapisec.default.svc.cluster.local"
                                                        },
                                                        "destination.service.uid":{
                                                            "string_value":"istio://default/services/goapisec"
                                                        },
                                                        "destination.service.name":{
                                                            "string_value":"goapisec"
                                                        },
                                                        "destination.service.namespace":{
                                                            "string_value":"default"
                                                        }
                                                    }
                                                },
                                                "mixer_attributes":{
                                                    "attributes":{
                                                        "destination.service.host":{
                                                            "string_value":"goapisec.default.svc.cluster.local"
                                                        },
                                                        "destination.service.uid":{
                                                            "string_value":"istio://default/services/goapisec"
                                                        },
                                                        "destination.service.namespace":{
                                                            "string_value":"default"
                                                        },
                                                        "destination.service.name":{
                                                            "string_value":"goapisec"
                                                        }
                                                    }
                                                }
                                            }
                                        }
                                    },
                                    {
                                        "match":{
                                            "prefix":"/"
                                        },
                                        "route":{
                                            "cluster":"outbound|81|v1|goapi.default.svc.cluster.local",
                                            "timeout":"0s",
                                            "retry_policy":{
                                                "retry_on":"connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
                                                "num_retries":2,
                                                "retry_host_predicate":[
                                                    {
                                                        "name":"envoy.retry_host_predicates.previous_hosts"
                                                    }
                                                ],
                                                "host_selection_retry_max_attempts":"5",
                                                "retriable_status_codes":[
                                                    503
                                                ]
                                            },
                                            "max_grpc_timeout":"0s"
                                        },
                                        "metadata":{
                                            "filter_metadata":{
                                                "istio":{
                                                    "config":"/apis/networking/v1alpha3/namespaces/default/virtual-service/goapi-default-xixi"
                                                }
                                            }
                                        },
                                        "decorator":{
                                            "operation":"goapi.default.svc.cluster.local:81/*"
                                        },
                                        "per_filter_config":{
                                            "mixer":{
                                                "disable_check_calls":true,
                                                "forward_attributes":{
                                                    "attributes":{
                                                        "destination.service.uid":{
                                                            "string_value":"istio://default/services/goapi"
                                                        },
                                                        "destination.service.host":{
                                                            "string_value":"goapi.default.svc.cluster.local"
                                                        },
                                                        "destination.service.namespace":{
                                                            "string_value":"default"
                                                        },
                                                        "destination.service.name":{
                                                            "string_value":"goapi"
                                                        }
                                                    }
                                                },
                                                "mixer_attributes":{
                                                    "attributes":{
                                                        "destination.service.namespace":{
                                                            "string_value":"default"
                                                        },
                                                        "destination.service.name":{
                                                            "string_value":"goapi"
                                                        },
                                                        "destination.service.host":{
                                                            "string_value":"goapi.default.svc.cluster.local"
                                                        },
                                                        "destination.service.uid":{
                                                            "string_value":"istio://default/services/goapi"
                                                        }
                                                    }
                                                }
                                            }
                                        }
                                    }
                                ]
                            }
                        ],
                        "validate_clusters":false
                    },
                    "last_updated":"2019-06-28T03:17:42.772Z"
                }
            ]
        }
    ]
}

参考文献 [1] 崔秀龙. 深入浅出 Istio | Service Mesh 快速入门与实践. 版次:2019年3月第1版 [2] 杨章显. Service Mesh 实战 | 基于 Linkerd 和 Kubernetes 的微服务实践. 版次:2019年1月第1版