Spring boot cannot connect to postgre database deployed on another kubernetes pod

I deployed my microservice hello-k8s in a pod and another pod as postgre-server, I have successfully connected to localhost using port-forward and try to create a table in the deployed database. But when I deploy hello-k8s it return error due to database connection attempt failed.

Disclaimer: kube-dns is working fine, I already try nslookup to the postgre-svc and it works well.

Here is the error

Caused by: java.net.UnknownHostException: postgre-svc
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220) ~[na:na] at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:609) ~[na:na]
at org.postgresql.core.PGStream.(PGStream.java:81) ~[postgresql-42.2.12.jar:42.2.12]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:93) ~[postgresql-42.2.12.jar:42.2.12]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197) ~[postgresql-42.2.12.jar:42.2.12]

Here is the application.yml file

spring:
  application:
    name: hello-k8s
  datasource:
    jdbc-url: jdbc:postgresql://${PG_SERVER}/first-db
    username: postgres
    password: password
    hikari:
      maximum-pool-size: 2
      connection-test-query: SELECT 1
    driver-class-name: org.postgresql.Driver
  jpa:
    hibernate:
      ddl-auto: update
    show-sql: true

Here is the configuration code to connect to database

@Configuration
@ConfigurationProperties(prefix = "spring.datasource")
@EnableJpaRepositories(entityManagerFactoryRef = "localEntityManagerFactory",
    transactionManagerRef = "localTransactionManager",
    basePackages = "id.hellok8s.repository")
public class JpaConfig extends HikariConfig{

    @Value("${spring.jpa.show-sql}")
    private boolean showSql;

    public JpaConfig(){}

    @Bean(name = "localDataSource")
    @Primary
    public DataSource dataSource(){
        return new HikariDataSource(this);
    }

    @Bean(name = "localEntityManagerFactory")
    @Primary
    public LocalContainerEntityManagerFactoryBean locaEntityManagerFactory(final EntityManagerFactoryBuilder builder, 
        @Qualifier("localDataSource") final DataSource dataSource){
            final LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
            entityManagerFactoryBean.setJpaVendorAdapter(this.vendorAdapter());
            entityManagerFactoryBean.setDataSource(dataSource);
            entityManagerFactoryBean.setPersistenceProviderClass(HibernatePersistenceProvider.class);
            entityManagerFactoryBean.setPackagesToScan("id.hellok8s.model");
            entityManagerFactoryBean.afterPropertiesSet();
            return entityManagerFactoryBean;
    }

    @Bean(name = "localTransactionManager")
    @Primary
    public PlatformTransactionManager localTransactionManager(
            @Qualifier("localEntityManagerFactory") final EntityManagerFactory emf){
        return new JpaTransactionManager(emf);
    }

    private HibernateJpaVendorAdapter vendorAdapter(){
        HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        vendorAdapter.setShowSql(showSql);
        return vendorAdapter;
    }
}

Here is the postgre service

apiVersion: v1
kind: Service
metadata:
  name: postgre-svc
spec:
  selector:
    app: postgre
  ports:
    - port: 5432
      targetPort: 5432
  type: ClusterIP

Here is my hello-k8s.yml deployment

apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: "hello-k8s"
  namespace: "hello-k8s"
  labels:
    app: "hello-k8s"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: "hello-k8s"
  template:
    metadata:
      labels:
        app: "hello-k8s"
    spec:
      containers:
      - name: hello-k8s
        image: [my-docker-container-registry]/hello-k8s:1.0.0-SNAPSHOT
        imagePullPolicy: Always
        env:
          - name: PG_SERVER
            value: "postgre-svc"

I have feeling it is due to HikariConfig, but I haven’t found any reference yet for that. anyway, anyone in the world has face issue like this? Feel blessed if anyone can help or have advice 🙂

Answer

Your client Deployment is explicitly labeled to deploy into a different namespace, but the database Service isn’t. By default this will cause the database Service to get deployed into the default namespace; the two objects being in different namespaces will cause the DNS issue you’re getting.

I typically don’t include an explicit namespace: in my Kubernetes YAML files. Instead, I use the kubectl --namespace option if I want to install things in a specific namespace. This also makes it a little easier to reuse a set of YAML files in a slightly different context.

You should also be able to make this work by pointing at the Service in the default namespace; set PG_SERVER to postgre-svc.default or postgres-svc.default.svc.cluster-local, including the other namespace name in the DNS name. (The database’s StatefulSet object also needs to be in the same namespace as its Service, so double-check that they’re correctly deployed together.)